Skip to contentSkip to conversation

net: annotate data-races around sysctl knobs

PATCH In review Jiayuan Chen 5 participants 14 messages Nov 18 – Nov 19

Subsystems (8)
lore
Share & tools
lore
#0 lore raw

v2 cover net: annotate data-races around sysctl knobs

This series annotates data races around a few sysctl knobs under net/. They are read without any lock while their handlers update them with WRITE_ONCE(), so add READ_ONCE() to the readers, and WRITE_ONCE() to the handlers and netlink paths that store to the same fields with plain writes.

There are no Fixes tags. Done one knob per patch this would be well over a hundred commits, so I grouped them by subsystem and feature instead to keep the review manageable. If maintainers prefer, I can split it up further with a Fixes tag on each one, as several smaller series.

No functional change.

v2:
- only resend the patches that got a Reviewed-by or Acked-by, with the tags picked up; they are unchanged
- netfilter patches are moved to a separate series for nf-next (Simon)
- calipso is sent separately as a fix
- the other patches had no review yet and are left out for now
v1: https://lore.kernel.org/netdev/[email protected]/

Jiayuan Chen (5):
inet: frags: annotate data-races around fqdir->timeout and

    fqdir->max_dist

ipv6: annotate data-races around icmpv6 reply sysctls
mptcp: annotate data-races around pernet sysctls
rds: annotate data-races around core rds sysctls net/smc: annotate data-races around smc sysctls

 include/net/ipv6.h       |  2 +-
 net/ipv4/inet_fragment.c |  2 +-
 net/ipv4/ip_fragment.c   |  6 +++---
 net/ipv6/icmp.c          | 10 +++++-----
 net/mptcp/ctrl.c         | 18 +++++++++---------
 net/rds/recv.c           |  2 +-
 net/rds/send.c           |  8 ++++----
 net/rds/threads.c        |  4 ++--
 net/smc/af_smc.c         |  8 ++++----
 net/smc/smc_clc.c        |  8 ++++----
 net/smc/smc_core.c       |  2 +-
 net/smc/smc_tx.c         |  2 +-
 12 files changed, 36 insertions(+), 36 deletions(-)
Signature · 1 line
2.43.0
#1 lore raw

PATCH v2 1/5 inet: frags: annotate data-races around fqdir->timeout and fqdir->max_dist

fqdir->timeout and fqdir->max_dist are read without any lock while their sysctl handlers update them with WRITE_ONCE(). Add READ_ONCE() to the readers.

---
 net/ipv4/inet_fragment.c | 2 +-
 net/ipv4/ip_fragment.c   | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)
net/ipv4/inet_fragment.c +1 -1
index f330d0a398233..8143162eb9f53 100644
@@ -416,7 +416,7 @@ static struct inet_frag_queue *inet_frag_create(struct fqdir *fqdir,
 		inet_frag_putn(q, 2);
 		return NULL;
 	}
-	mod_timer(&q->timer, jiffies + fqdir->timeout);
+	mod_timer(&q->timer, jiffies + READ_ONCE(fqdir->timeout));
 	spin_unlock_bh(&q->lock);
 	return q;
 }
net/ipv4/ip_fragment.c +3 -3
index ef7fd61284a3f..5ac69d716da2a 100644
@@ -89,7 +89,7 @@ static void ip4_frag_init(struct inet_frag_queue *q, const void *a)
 
 	q->key.v4 = *key;
 	qp->ecn = 0;
-	if (q->fqdir->max_dist) {
+	if (READ_ONCE(q->fqdir->max_dist)) {
 		rcu_read_lock();
 		p = inet_getpeer_v4(net->ipv4.peers, key->saddr, key->vif);
 		if (p && !refcount_inc_not_zero(&p->refcnt))
@@ -220,7 +220,7 @@ static struct ipq *ip_find(struct net *net, struct iphdr *iph,
 static int ip_frag_too_far(struct ipq *qp)
 {
 	struct inet_peer *peer = qp->peer;
-	unsigned int max = qp->q.fqdir->max_dist;
+	unsigned int max = READ_ONCE(qp->q.fqdir->max_dist);
 	unsigned int start, end;
 
 	int rc;
@@ -242,7 +242,7 @@ static int ip_frag_too_far(struct ipq *qp)
 
 static int ip_frag_reinit(struct ipq *qp)
 {
-	if (!mod_timer_pending(&qp->q.timer, jiffies + qp->q.fqdir->timeout))
+	if (!mod_timer_pending(&qp->q.timer, jiffies + READ_ONCE(qp->q.fqdir->timeout)))
 		return -ETIMEDOUT;
 
 	inet_frag_queue_flush(&qp->q, SKB_DROP_REASON_FRAG_TOO_FAR);
Signature · 1 line
2.43.0
#2 lore raw

Re: [PATCH net-next v2 1/5] inet: frags: annotate data-races around fqdir->timeout and fqdir->max_dist

Sashiko has reviewed this patch and found no issues.
Quoted 5 lines

fqdir->timeout and fqdir->max_dist are read without any lock while their sysctl handlers update them with WRITE_ONCE(). Add READ_ONCE() to the readers.

Sashiko has reviewed this patch and found no issues.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
#3 lore raw

PATCH v2 2/5 ipv6: annotate data-races around icmpv6 reply sysctls

In reply to Jiayuan Chen

icmpv6_echo_ignore_all, icmpv6_echo_ignore_multicast, icmpv6_echo_ignore_anycast, anycast_src_echo_reply and fwmark_reflect are read without any lock while their sysctl handlers update them with WRITE_ONCE(). Add READ_ONCE() to the readers.

KCSAN reports it when net.ipv6.icmp.echo_ignore_all is changed while ICMPv6 echo requests are received:

  BUG: KCSAN: data-race in icmpv6_rcv / proc_dou8vec_minmax

write (marked) to 0xffffffff8ee2940d of 1 bytes by task 504 on cpu 0:

   proc_dou8vec_minmax+0x1ab/0x1f0
   proc_sys_call_handler+0x282/0x3a0
   proc_sys_write+0x13/0x20
   ...

read to 0xffffffff8ee2940d of 1 bytes by interrupt on cpu 3:

   icmpv6_rcv+0x99d/0xd40
   ip6_protocol_deliver_rcu+0x787/0x9a0
   ip6_input_finish+0xb0/0x1b0
   ip6_input+0xa8/0x250
   ipv6_rcv+0x14b/0x1c0
   __netif_receive_skb_one_core+0x8f/0xe0
   ...

value changed: 0x01 -> 0x00

---
 include/net/ipv6.h |  2 +-
 net/ipv6/icmp.c    | 10 +++++-----
 2 files changed, 6 insertions(+), 6 deletions(-)
include/net/ipv6.h +1 -1
index 579df1516d251..b77dc5c3ea268 100644
@@ -199,7 +199,7 @@ struct sk_buff *ip6_frag_next(struct sk_buff *skb,
 			      struct ip6_frag_state *state);
 
 #define IP6_REPLY_MARK(net, mark) \
-	((net)->ipv6.sysctl.fwmark_reflect ? (mark) : 0)
+	(READ_ONCE((net)->ipv6.sysctl.fwmark_reflect) ? (mark) : 0)
 
 #include <net/sock.h>
 
net/ipv6/icmp.c +5 -5
index 4122ce3055e49..abfe2893c18f4 100644
@@ -946,17 +946,17 @@ static enum skb_drop_reason icmpv6_echo_reply(struct sk_buff *skb)
 	u8 type;
 
 	if (ipv6_addr_is_multicast(&ipv6_hdr(skb)->daddr) &&
-	    net->ipv6.sysctl.icmpv6_echo_ignore_multicast)
+	    READ_ONCE(net->ipv6.sysctl.icmpv6_echo_ignore_multicast))
 		return reason;
 
 	saddr = &ipv6_hdr(skb)->daddr;
 
 	acast = ipv6_anycast_destination(skb_dst(skb), saddr);
-	if (acast && net->ipv6.sysctl.icmpv6_echo_ignore_anycast)
+	if (acast && READ_ONCE(net->ipv6.sysctl.icmpv6_echo_ignore_anycast))
 		return reason;
 
 	if (!ipv6_unicast_destination(skb) &&
-	    !(net->ipv6.sysctl.anycast_src_echo_reply && acast))
+	    !(READ_ONCE(net->ipv6.sysctl.anycast_src_echo_reply) && acast))
 		saddr = NULL;
 
 	if (icmph->icmp6_type == ICMPV6_EXT_ECHO_REQUEST)
@@ -1160,11 +1160,11 @@ static int icmpv6_rcv(struct sk_buff *skb)
 
 	switch (type) {
 	case ICMPV6_ECHO_REQUEST:
-		if (!net->ipv6.sysctl.icmpv6_echo_ignore_all)
+		if (!READ_ONCE(net->ipv6.sysctl.icmpv6_echo_ignore_all))
 			reason = icmpv6_echo_reply(skb);
 		break;
 	case ICMPV6_EXT_ECHO_REQUEST:
-		if (!net->ipv6.sysctl.icmpv6_echo_ignore_all &&
+		if (!READ_ONCE(net->ipv6.sysctl.icmpv6_echo_ignore_all) &&
 		    READ_ONCE(net->ipv4.sysctl_icmp_echo_enable_probe))
 			reason = icmpv6_echo_reply(skb);
 		break;
Signature · 1 line
2.43.0
#4 lore raw

Re: [PATCH net-next v2 2/5] ipv6: annotate data-races around icmpv6 reply sysctls

Sashiko has reviewed this patch and found no issues.
Quoted 25 lines

icmpv6_echo_ignore_all, icmpv6_echo_ignore_multicast, icmpv6_echo_ignore_anycast, anycast_src_echo_reply and fwmark_reflect are read without any lock while their sysctl handlers update them with WRITE_ONCE(). Add READ_ONCE() to the readers.

KCSAN reports it when net.ipv6.icmp.echo_ignore_all is changed while ICMPv6 echo requests are received:

  BUG: KCSAN: data-race in icmpv6_rcv / proc_dou8vec_minmax

write (marked) to 0xffffffff8ee2940d of 1 bytes by task 504 on cpu 0:

   proc_dou8vec_minmax+0x1ab/0x1f0
   proc_sys_call_handler+0x282/0x3a0
   proc_sys_write+0x13/0x20
   ...

read to 0xffffffff8ee2940d of 1 bytes by interrupt on cpu 3:

   icmpv6_rcv+0x99d/0xd40
   ip6_protocol_deliver_rcu+0x787/0x9a0
   ip6_input_finish+0xb0/0x1b0
   ip6_input+0xa8/0x250
   ipv6_rcv+0x14b/0x1c0
   __netif_receive_skb_one_core+0x8f/0xe0
   ...
Quoted 3 lines

value changed: 0x01 -> 0x00

Sashiko has reviewed this patch and found no issues.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
#5 lore raw

PATCH v2 3/5 mptcp: annotate data-races around pernet sysctls

In reply to Jiayuan Chen

enabled, add_addr_timeout, checksum_enabled, allow_join_initial_addr_port, stale_loss_cnt, close_timeout, pm_type and syn_retrans_before_tcp_fallback are read by their ctrl.c helpers without any lock while the sysctl handlers update them with WRITE_ONCE(), and the path_manager handler stores pm_type with a plain write. Add READ_ONCE() to the readers and WRITE_ONCE() to that handler.

KCSAN reports it when net.mptcp.enabled is changed while MPTCP sockets are being created:

  BUG: KCSAN: data-race in mptcp_is_enabled / proc_dou8vec_minmax

write (marked) to 0xffff8d724248abc9 of 1 bytes by task 733 on cpu 3:

   proc_dou8vec_minmax+0x1ab/0x1f0
   proc_sys_call_handler+0x282/0x3a0
   proc_sys_write+0x13/0x20
   ...

read to 0xffff8d724248abc9 of 1 bytes by task 735 on cpu 1:

   mptcp_is_enabled+0x33/0x50
   mptcp_init_sock+0x35/0x300
   inet_create+0x4a6/0x8e0
   ...

value changed: 0x00 -> 0x01

---
 net/mptcp/ctrl.c | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)
net/mptcp/ctrl.c +9 -9
index 63c5747f0f63c..164f79b9deb0e 100644
@@ -50,39 +50,39 @@ static struct mptcp_pernet *mptcp_get_pernet(const struct net *net)
 
 int mptcp_is_enabled(const struct net *net)
 {
-	return mptcp_get_pernet(net)->mptcp_enabled;
+	return READ_ONCE(mptcp_get_pernet(net)->mptcp_enabled);
 }
 
 unsigned int mptcp_get_add_addr_timeout(const struct net *net)
 {
-	return mptcp_get_pernet(net)->add_addr_timeout;
+	return READ_ONCE(mptcp_get_pernet(net)->add_addr_timeout);
 }
 
 int mptcp_is_checksum_enabled(const struct net *net)
 {
-	return mptcp_get_pernet(net)->checksum_enabled;
+	return READ_ONCE(mptcp_get_pernet(net)->checksum_enabled);
 }
 
 int mptcp_allow_join_id0(const struct net *net)
 {
-	return mptcp_get_pernet(net)->allow_join_initial_addr_port;
+	return READ_ONCE(mptcp_get_pernet(net)->allow_join_initial_addr_port);
 }
 
 unsigned int mptcp_stale_loss_cnt(const struct net *net)
 {
-	return mptcp_get_pernet(net)->stale_loss_cnt;
+	return READ_ONCE(mptcp_get_pernet(net)->stale_loss_cnt);
 }
 
 unsigned int mptcp_close_timeout(const struct sock *sk)
 {
 	if (sock_flag(sk, SOCK_DEAD))
 		return TCP_TIMEWAIT_LEN;
-	return mptcp_get_pernet(sock_net(sk))->close_timeout;
+	return READ_ONCE(mptcp_get_pernet(sock_net(sk))->close_timeout);
 }
 
 int mptcp_get_pm_type(const struct net *net)
 {
-	return mptcp_get_pernet(net)->pm_type;
+	return READ_ONCE(mptcp_get_pernet(net)->pm_type);
 }
 
 const char *mptcp_get_path_manager(const struct net *net)
@@ -230,7 +230,7 @@ static int proc_path_manager(const struct ctl_table *ctl, int write,
 				pm_type = MPTCP_PM_TYPE_KERNEL;
 			else if (strncmp(pm_name, "userspace", MPTCP_PM_NAME_MAX) == 0)
 				pm_type = MPTCP_PM_TYPE_USERSPACE;
-			pernet->pm_type = pm_type;
+			WRITE_ONCE(pernet->pm_type, pm_type);
 		}
 	}
 
@@ -551,7 +551,7 @@ void mptcp_active_detect_blackhole(struct sock *ssk, bool expired)
 
 	net = sock_net(ssk);
 	timeouts = inet_csk(ssk)->icsk_retransmits;
-	to_max = mptcp_get_pernet(net)->syn_retrans_before_tcp_fallback;
+	to_max = READ_ONCE(mptcp_get_pernet(net)->syn_retrans_before_tcp_fallback);
 
 	if (timeouts == to_max || (timeouts < to_max && expired)) {
 		subflow->mpc_drop = 1;
Signature · 1 line
2.43.0
#6 lore raw

Re: [PATCH net-next v2 3/5] mptcp: annotate data-races around pernet sysctls

Sashiko has reviewed this patch and found no issues.
Quoted 19 lines

enabled, add_addr_timeout, checksum_enabled, allow_join_initial_addr_port, stale_loss_cnt, close_timeout, pm_type and syn_retrans_before_tcp_fallback are read by their ctrl.c helpers without any lock while the sysctl handlers update them with WRITE_ONCE(), and the path_manager handler stores pm_type with a plain write. Add READ_ONCE() to the readers and WRITE_ONCE() to that handler.

KCSAN reports it when net.mptcp.enabled is changed while MPTCP sockets are being created:

  BUG: KCSAN: data-race in mptcp_is_enabled / proc_dou8vec_minmax

write (marked) to 0xffff8d724248abc9 of 1 bytes by task 733 on cpu 3:

   proc_dou8vec_minmax+0x1ab/0x1f0
   proc_sys_call_handler+0x282/0x3a0
   proc_sys_write+0x13/0x20
   ...

read to 0xffff8d724248abc9 of 1 bytes by task 735 on cpu 1:

Quoted 8 lines
   mptcp_is_enabled+0x33/0x50
   mptcp_init_sock+0x35/0x300
   inet_create+0x4a6/0x8e0
   ...

value changed: 0x00 -> 0x01

Sashiko has reviewed this patch and found no issues.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
#7 lore raw

PATCH v2 4/5 rds: annotate data-races around core rds sysctls

In reply to Jiayuan Chen

reconnect_min_delay_ms, reconnect_max_delay_ms, max_unacked_packets, max_unacked_bytes and ping_enable are read without any lock while their sysctl handlers update them with WRITE_ONCE(). Add READ_ONCE() to the readers.

---
 net/rds/recv.c    | 2 +-
 net/rds/send.c    | 8 ++++----
 net/rds/threads.c | 4 ++--
 3 files changed, 7 insertions(+), 7 deletions(-)
net/rds/recv.c +1 -1
index 00ab60f2d9969..daccbfc8cb8a2 100644
@@ -370,7 +370,7 @@ void rds_recv_incoming(struct rds_connection *conn, struct in6_addr *saddr,
 	}
 	cp->cp_next_rx_seq = be64_to_cpu(inc->i_hdr.h_sequence) + 1;
 
-	if (rds_sysctl_ping_enable && inc->i_hdr.h_dport == 0) {
+	if (READ_ONCE(rds_sysctl_ping_enable) && inc->i_hdr.h_dport == 0) {
 		if (inc->i_hdr.h_sport == 0) {
 			rdsdebug("ignore ping with 0 sport from %pI6c\n",
 				 saddr);
net/rds/send.c +4 -4
index f38967f65af18..66e75a382bcc7 100644
@@ -87,8 +87,8 @@ void rds_send_path_reset(struct rds_conn_path *cp)
 
 	cp->cp_conn->c_map_queued = 0;
 
-	cp->cp_unacked_packets = rds_sysctl_max_unacked_packets;
-	cp->cp_unacked_bytes = rds_sysctl_max_unacked_bytes;
+	cp->cp_unacked_packets = READ_ONCE(rds_sysctl_max_unacked_packets);
+	cp->cp_unacked_bytes = READ_ONCE(rds_sysctl_max_unacked_bytes);
 
 	/* Mark messages as retransmissions, and move them to the send q */
 	spin_lock_irqsave(&cp->cp_lock, flags);
@@ -380,9 +380,9 @@ int rds_send_xmit(struct rds_conn_path *cp)
 				set_bit(RDS_MSG_ACK_REQUIRED, &rm->m_flags);
 
 				cp->cp_unacked_packets =
-					rds_sysctl_max_unacked_packets;
+					READ_ONCE(rds_sysctl_max_unacked_packets);
 				cp->cp_unacked_bytes =
-					rds_sysctl_max_unacked_bytes;
+					READ_ONCE(rds_sysctl_max_unacked_bytes);
 				rds_stats_inc(s_send_ack_required);
 			} else {
 				cp->cp_unacked_bytes -= len;
net/rds/threads.c +2 -2
index 639302bab51ef..115fcff0a5b13 100644
@@ -137,7 +137,7 @@ void rds_queue_reconnect(struct rds_conn_path *cp)
 
 	set_bit(RDS_RECONNECT_PENDING, &cp->cp_flags);
 	if (cp->cp_reconnect_jiffies == 0) {
-		cp->cp_reconnect_jiffies = rds_sysctl_reconnect_min_jiffies;
+		cp->cp_reconnect_jiffies = READ_ONCE(rds_sysctl_reconnect_min_jiffies);
 		rcu_read_lock();
 		if (!rds_destroy_pending(cp->cp_conn))
 			queue_delayed_work(cp->cp_wq, &cp->cp_conn_w, 0);
@@ -156,7 +156,7 @@ void rds_queue_reconnect(struct rds_conn_path *cp)
 	rcu_read_unlock();
 
 	cp->cp_reconnect_jiffies = min(cp->cp_reconnect_jiffies * 2,
-					rds_sysctl_reconnect_max_jiffies);
+					READ_ONCE(rds_sysctl_reconnect_max_jiffies));
 }
 
 void rds_connect_worker(struct work_struct *work)
Signature · 1 line
2.43.0
#8 lore raw

Re: [PATCH net-next v2 4/5] rds: annotate data-races around core rds sysctls

Sashiko has reviewed this patch and found no issues.
Quoted 6 lines

reconnect_min_delay_ms, reconnect_max_delay_ms, max_unacked_packets, max_unacked_bytes and ping_enable are read without any lock while their sysctl handlers update them with WRITE_ONCE(). Add READ_ONCE() to the readers.

Sashiko has reviewed this patch and found no issues.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
#9 lore raw

Re: [PATCH net-next v2 4/5] rds: annotate data-races around core rds sysctls

In reply to Jiayuan Chen
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider.

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider.

Critical: 0 · High: 0 · Medium: 1 · Low: 0

If you believe none of the issues are legitimate, please reply to each issue of Medium severity or higher and explain why the AI is wrong. If any of them are legitimate and you plan to respin, please let patchwork know by including "pw-bot: cr" as a separate line at the end of your reply (one such reply per series is enough).

- [Medium] The patch only annotates the RDS datapath readers of rds_sysctl_reconnect_min_jiffies and rds_sysctl_reconnect_max_jiffies.

--

rds: annotate data-races around core rds sysctls

This patch adds READ_ONCE() to the lockless RDS readers of reconnect_min_delay_ms, reconnect_max_delay_ms, max_unacked_packets, max_unacked_bytes and ping_enable. The sysctl handlers already update these with WRITE_ONCE().

[ ... ]

Quoted 14 lines
net/rds/threads.c +1 -1
index 639302bab51ef..115fcff0a5b13 100644
@@ -137,7 +137,7 @@ void rds_queue_reconnect(struct rds_conn_path *cp)
 
 	set_bit(RDS_RECONNECT_PENDING, &cp->cp_flags);
 	if (cp->cp_reconnect_jiffies == 0) {
-		cp->cp_reconnect_jiffies = rds_sysctl_reconnect_min_jiffies;
+		cp->cp_reconnect_jiffies = READ_ONCE(rds_sysctl_reconnect_min_jiffies);
 		rcu_read_lock();
 		if (!rds_destroy_pending(cp->cp_conn))
 			queue_delayed_work(cp->cp_wq, &cp->cp_conn_w, 0);

net/rds/threads.c void rds_queue_reconnect(struct rds_conn_path *cp) · line 160

Quoted 6 lines
net/rds/threads.c +1 -1
@@ -156,7 +156,7 @@ void rds_queue_reconnect(struct rds_conn_path *cp)
 	rcu_read_unlock();
 
 	cp->cp_reconnect_jiffies = min(cp->cp_reconnect_jiffies * 2,
-					rds_sysctl_reconnect_max_jiffies);
+					READ_ONCE(rds_sysctl_reconnect_max_jiffies));
 }

[Severity: Medium]
These two variables have one more lockless reader that is still a plain
load: the sysctl handlers themselves. Does that leave part of the race described in the commit message unfixed?

In net/rds/sysctl.c, each reconnect sysctl uses the other one's variable as its range bound:

net/rds/sysctl.c:rds_sysctl_rds_table[] {
.procname = "reconnect_min_delay_ms",
.data = &rds_sysctl_reconnect_min_jiffies,
...
.extra2 = &rds_sysctl_reconnect_max_jiffies,
...
.procname = "reconnect_max_delay_ms",
.data = &rds_sysctl_reconnect_max_jiffies,
...
.extra1 = &rds_sysctl_reconnect_min_jiffies,
}

Both entries use proc_doulongvec_ms_jiffies_minmax(), which ends up in proc_ulong_conv() in kernel/sysctl.c:

kernel/sysctl.c:proc_ulong_conv() {
	...
		if ((tbl->extra1 && *(ulong *)tbl->extra1 > tmp_k) ||
		    (tbl->extra2 && *(ulong *)tbl->extra2 < tmp_k))
			return -ERANGE;
		WRITE_ONCE(*k_ptr, tmp_k);
	...
}

If one task writes reconnect_min_delay_ms while another writes reconnect_max_delay_ms, each handler's WRITE_ONCE() races with the other handler's plain read of the same variable. KCSAN would report this.

The bound check is also not atomic with the store, so both writers can pass their checks against the other's old value. For example, with min=100 and max=1000:

task A: write min=500, checked against max=1000, passes task B: write max=200, checked against min=100, passes

Both writes succeed and leave min=500 > max=200. rds_queue_reconnect() then starts the backoff at READ_ONCE(min) = 500 and caps the next value at min(1000, READ_ONCE(max)) = 200. That puts the backoff below the configured minimum.

Could the extra1/extra2 loads in proc_ulong_conv() use READ_ONCE()? Or could the two RDS handlers be serialized so that min <= max is enforced atomically? As far as I can tell, no later patch in the series changes kernel/sysctl.c or net/rds/sysctl.c.

Signature · 1 line
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20261118045137.174026-1-jiayuan.chen%40linux.dev
#10 lore raw

PATCH v2 5/5 net/smc: annotate data-races around smc sysctls

In reply to Jiayuan Chen

autocorking_size, smcr_buf_type, smcr_max_links_per_lgr, smcr_max_conns_per_lgr and limit_smc_hs are read without any lock while their sysctl handlers update them with WRITE_ONCE(), and the SMC netlink handlers toggle limit_smc_hs with plain writes. Add READ_ONCE() to the readers and WRITE_ONCE() to those handlers.

---
 net/smc/af_smc.c   | 8 ++++----
 net/smc/smc_clc.c  | 8 ++++----
 net/smc/smc_core.c | 2 +-
 net/smc/smc_tx.c   | 2 +-
 4 files changed, 10 insertions(+), 10 deletions(-)
net/smc/af_smc.c +4 -4
index bfc0c7e71a12b..1d3eae064fe0d 100644
@@ -88,7 +88,7 @@ int smc_nl_dump_hs_limitation(struct sk_buff *skb, struct netlink_callback *cb)
 		return -ENOMEM;
 
 	if (nla_put_u8(skb, SMC_NLA_HS_LIMITATION_ENABLED,
-		       sock_net(skb->sk)->smc.limit_smc_hs))
+		       READ_ONCE(sock_net(skb->sk)->smc.limit_smc_hs)))
 		goto err;
 
 	genlmsg_end(skb, hdr);
@@ -102,13 +102,13 @@ int smc_nl_dump_hs_limitation(struct sk_buff *skb, struct netlink_callback *cb)
 
 int smc_nl_enable_hs_limitation(struct sk_buff *skb, struct genl_info *info)
 {
-	sock_net(skb->sk)->smc.limit_smc_hs = true;
+	WRITE_ONCE(sock_net(skb->sk)->smc.limit_smc_hs, true);
 	return 0;
 }
 
 int smc_nl_disable_hs_limitation(struct sk_buff *skb, struct genl_info *info)
 {
-	sock_net(skb->sk)->smc.limit_smc_hs = false;
+	WRITE_ONCE(sock_net(skb->sk)->smc.limit_smc_hs, false);
 	return 0;
 }
 
@@ -411,7 +411,7 @@ void smc_sk_init(struct net *net, struct sock *sk, int protocol)
 	spin_lock_init(&smc->conn.send_lock);
 	mutex_init(&smc->clcsock_release_lock);
 	smc_init_saved_callbacks(smc);
-	smc->limit_smc_hs = net->smc.limit_smc_hs;
+	smc->limit_smc_hs = READ_ONCE(net->smc.limit_smc_hs);
 	smc->use_fallback = false; /* assume rdma capability first */
 	smc->fallback_rsn = 0;
 	smc_close_init(smc);
net/smc/smc_clc.c +4 -4
index 014d527d5462d..df0afeef67cbd 100644
@@ -990,8 +990,8 @@ int smc_clc_send_proposal(struct smc_sock *smc, struct smc_init_info *ini)
 	}
 	if (smcr_indicated(ini->smc_type_v2)) {
 		memcpy(v2_ext->roce, ini->smcrv2.ib_gid_v2, SMC_GID_SIZE);
-		v2_ext->max_conns = net->smc.sysctl_max_conns_per_lgr;
-		v2_ext->max_links = net->smc.sysctl_max_links_per_lgr;
+		v2_ext->max_conns = READ_ONCE(net->smc.sysctl_max_conns_per_lgr);
+		v2_ext->max_links = READ_ONCE(net->smc.sysctl_max_links_per_lgr);
 	}
 
 	pclc_base->hdr.length = htons(plen);
@@ -1272,12 +1272,12 @@ int smc_clc_srv_v2x_features_validate(struct smc_sock *smc,
 
 	if (ini->smcr_version & SMC_V2) {
 		ini->max_conns = min_t(u8, pclc_v2_ext->max_conns,
-				       net->smc.sysctl_max_conns_per_lgr);
+				       READ_ONCE(net->smc.sysctl_max_conns_per_lgr));
 		if (ini->max_conns < SMC_CONN_PER_LGR_MIN)
 			return SMC_CLC_DECL_MAXCONNERR;
 
 		ini->max_links = min_t(u8, pclc_v2_ext->max_links,
-				       net->smc.sysctl_max_links_per_lgr);
+				       READ_ONCE(net->smc.sysctl_max_links_per_lgr));
 		if (ini->max_links < SMC_LINKS_ADD_LNK_MIN)
 			return SMC_CLC_DECL_MAXLINKERR;
 	}
net/smc/smc_core.c +1 -1
index a3164201c1420..2db3a87054835 100644
@@ -1010,7 +1010,7 @@ static int smc_lgr_create(struct smc_sock *smc, struct smc_init_info *ini)
 		lgr->net = smc_ib_net(lnk->smcibdev);
 		lgr_list = &smc_lgr_list.list;
 		lgr_lock = &smc_lgr_list.lock;
-		lgr->buf_type = lgr->net->smc.sysctl_smcr_buf_type;
+		lgr->buf_type = READ_ONCE(lgr->net->smc.sysctl_smcr_buf_type);
 		atomic_inc(&lgr_cnt);
 	}
 	smc->conn.lgr = lgr;
net/smc/smc_tx.c +1 -1
index 0f0c6df3bf2ce..30bffe79d06a3 100644
@@ -148,7 +148,7 @@ static bool smc_should_autocork(struct smc_sock *smc)
 	int corking_size;
 
 	corking_size = min_t(unsigned int, conn->sndbuf_desc->len >> 1,
-			     sock_net(&smc->sk)->smc.sysctl_autocorking_size);
+			     READ_ONCE(sock_net(&smc->sk)->smc.sysctl_autocorking_size));
 
 	if (atomic_read(&conn->cdc_pend_tx_wr) == 0 ||
 	    smc_tx_prepared_sends(conn) > corking_size)
Signature · 1 line
2.43.0
#11 lore raw

Re: [PATCH net-next v2 5/5] net/smc: annotate data-races around smc sysctls

Sashiko has reviewed this patch and found no issues.
Quoted 7 lines

autocorking_size, smcr_buf_type, smcr_max_links_per_lgr, smcr_max_conns_per_lgr and limit_smc_hs are read without any lock while their sysctl handlers update them with WRITE_ONCE(), and the SMC netlink handlers toggle limit_smc_hs with plain writes. Add READ_ONCE() to the readers and WRITE_ONCE() to those handlers.

Sashiko has reviewed this patch and found no issues.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
maintainer Reviewed-by on v2 5/5
#12 lore raw

Re: [PATCH net-next v2 5/5] net/smc: annotate data-races around smc sysctls

In reply to Jiayuan Chen
Quoted 101 lines · Jiayuan Chen

autocorking_size, smcr_buf_type, smcr_max_links_per_lgr, smcr_max_conns_per_lgr and limit_smc_hs are read without any lock while their sysctl handlers update them with WRITE_ONCE(), and the SMC netlink handlers toggle limit_smc_hs with plain writes. Add READ_ONCE() to the readers and WRITE_ONCE() to those handlers.

---
 net/smc/af_smc.c   | 8 ++++----
 net/smc/smc_clc.c  | 8 ++++----
 net/smc/smc_core.c | 2 +-
 net/smc/smc_tx.c   | 2 +-
 4 files changed, 10 insertions(+), 10 deletions(-)
net/smc/af_smc.c +4 -4
index bfc0c7e71a12b..1d3eae064fe0d 100644
@@ -88,7 +88,7 @@ int smc_nl_dump_hs_limitation(struct sk_buff *skb, struct netlink_callback *cb)
 		return -ENOMEM;
 
 	if (nla_put_u8(skb, SMC_NLA_HS_LIMITATION_ENABLED,
-		       sock_net(skb->sk)->smc.limit_smc_hs))
+		       READ_ONCE(sock_net(skb->sk)->smc.limit_smc_hs)))
 		goto err;
 
 	genlmsg_end(skb, hdr);
@@ -102,13 +102,13 @@ int smc_nl_dump_hs_limitation(struct sk_buff *skb, struct netlink_callback *cb)
 
 int smc_nl_enable_hs_limitation(struct sk_buff *skb, struct genl_info *info)
 {
-	sock_net(skb->sk)->smc.limit_smc_hs = true;
+	WRITE_ONCE(sock_net(skb->sk)->smc.limit_smc_hs, true);
 	return 0;
 }
 
 int smc_nl_disable_hs_limitation(struct sk_buff *skb, struct genl_info *info)
 {
-	sock_net(skb->sk)->smc.limit_smc_hs = false;
+	WRITE_ONCE(sock_net(skb->sk)->smc.limit_smc_hs, false);
 	return 0;
 }
 
@@ -411,7 +411,7 @@ void smc_sk_init(struct net *net, struct sock *sk, int protocol)
 	spin_lock_init(&smc->conn.send_lock);
 	mutex_init(&smc->clcsock_release_lock);
 	smc_init_saved_callbacks(smc);
-	smc->limit_smc_hs = net->smc.limit_smc_hs;
+	smc->limit_smc_hs = READ_ONCE(net->smc.limit_smc_hs);
 	smc->use_fallback = false; /* assume rdma capability first */
 	smc->fallback_rsn = 0;
 	smc_close_init(smc);
net/smc/smc_clc.c +4 -4
index 014d527d5462d..df0afeef67cbd 100644
@@ -990,8 +990,8 @@ int smc_clc_send_proposal(struct smc_sock *smc, struct smc_init_info *ini)
 	}
 	if (smcr_indicated(ini->smc_type_v2)) {
 		memcpy(v2_ext->roce, ini->smcrv2.ib_gid_v2, SMC_GID_SIZE);
-		v2_ext->max_conns = net->smc.sysctl_max_conns_per_lgr;
-		v2_ext->max_links = net->smc.sysctl_max_links_per_lgr;
+		v2_ext->max_conns = READ_ONCE(net->smc.sysctl_max_conns_per_lgr);
+		v2_ext->max_links = READ_ONCE(net->smc.sysctl_max_links_per_lgr);
 	}
 
 	pclc_base->hdr.length = htons(plen);
@@ -1272,12 +1272,12 @@ int smc_clc_srv_v2x_features_validate(struct smc_sock *smc,
 
 	if (ini->smcr_version & SMC_V2) {
 		ini->max_conns = min_t(u8, pclc_v2_ext->max_conns,
-				       net->smc.sysctl_max_conns_per_lgr);
+				       READ_ONCE(net->smc.sysctl_max_conns_per_lgr));
 		if (ini->max_conns < SMC_CONN_PER_LGR_MIN)
 			return SMC_CLC_DECL_MAXCONNERR;
 
 		ini->max_links = min_t(u8, pclc_v2_ext->max_links,
-				       net->smc.sysctl_max_links_per_lgr);
+				       READ_ONCE(net->smc.sysctl_max_links_per_lgr));
 		if (ini->max_links < SMC_LINKS_ADD_LNK_MIN)
 			return SMC_CLC_DECL_MAXLINKERR;
 	}
net/smc/smc_core.c +1 -1
index a3164201c1420..2db3a87054835 100644
@@ -1010,7 +1010,7 @@ static int smc_lgr_create(struct smc_sock *smc, struct smc_init_info *ini)
 		lgr->net = smc_ib_net(lnk->smcibdev);
 		lgr_list = &smc_lgr_list.list;
 		lgr_lock = &smc_lgr_list.lock;
-		lgr->buf_type = lgr->net->smc.sysctl_smcr_buf_type;
+		lgr->buf_type = READ_ONCE(lgr->net->smc.sysctl_smcr_buf_type);
 		atomic_inc(&lgr_cnt);
 	}
 	smc->conn.lgr = lgr;
net/smc/smc_tx.c +0 -0
index 0f0c6df3bf2ce..30bffe79d06a3 100644

net/smc/smc_tx.c static bool smc_should_autocork(struct smc_sock *smc) · line 154

Quoted 8 lines
net/smc/smc_tx.c +1 -1
@@ -148,7 +148,7 @@ static bool smc_should_autocork(struct smc_sock *smc)
 	int corking_size;
 
 	corking_size = min_t(unsigned int, conn->sndbuf_desc->len >> 1,
-			     sock_net(&smc->sk)->smc.sysctl_autocorking_size);
+			     READ_ONCE(sock_net(&smc->sk)->smc.sysctl_autocorking_size));
 
 	if (atomic_read(&conn->cdc_pend_tx_wr) == 0 ||
 	    smc_tx_prepared_sends(conn) > corking_size)

Regards,
Sidraya

#13 lore raw
In reply to Jiayuan Chen

Hi Jiayuan,

Thank you for your modifications, that's great!

Our CI did some validations and here is its report:

- KVM Validation: normal (except selftest_mptcp_join): Success! ✅
- KVM Validation: normal (only selftest_mptcp_join): Success! ✅
- KVM Validation: debug (except selftest_mptcp_join): Success! ✅
- KVM Validation: debug (only selftest_mptcp_join): Success! ✅
- KVM Validation: btf-normal (only bpftest_all): Success! ✅
- KVM Validation: btf-debug (only bpftest_all): Success! ✅
- Perf: Unstable: 2 failed test(s): error:sched-mem-pressure validation:sched-weak-secondary ⚠️
- Task: https://github.com/multipath-tcp/mptcp_net-next/actions/runs/44242779577

Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/c03ca5be7367
Patchwork: https://patchwork.kernel.org/project/mptcp/list/?series=1214545

If there are some issues, you can reproduce them using the same environment as the one used by the CI thanks to a docker image, e.g.:

    $ cd [kernel source code]
    $ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
        --pull always mptcp/mptcp-upstream-virtme-docker:latest \
        auto-normal

For more details:

https://github.com/multipath-tcp/mptcp-upstream-virtme-docker

Please note that despite all the efforts that have been already done to have a stable tests suite when executed on a public CI like here, it is possible some reported issues are not due to your modifications. Still, do not hesitate to help us improve that ;-)

Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (NGI0 Core)