net: annotate data-races
Subsystems (15)
cover net: annotate data-races
This series was inspired by a syzbot/KCSAN report.
This will later also permit some optimizations, like not having to lock the socket while reading/writing some of its fields.
Eric Dumazet (11):
net: annotate data-races around sk->sk_reserved_mem
net: annotate data-race around sk->sk_txrehash
net: annotate data-races around sk->sk_max_pacing_rate
net: add missing READ_ONCE(sk->sk_rcvlowat) annotation
net: annotate data-races around sk->sk_{rcv|snd}timeo
net: add missing READ_ONCE(sk->sk_sndbuf) annotation
net: add missing READ_ONCE(sk->sk_rcvbuf) annotation
net: annotate data-races around sk->sk_mark
net: add missing data-race annotations around sk->sk_peek_off
net: add missing data-race annotation for sk_ll_usec
net: annotate data-races around sk->sk_priority
include/net/inet_sock.h | 7 ++-- include/net/ip.h | 2 +- include/net/route.h | 4 +-- net/can/raw.c | 2 +- net/core/sock.c | 69 ++++++++++++++++++++++---------------- net/dccp/ipv6.c | 4 +-- net/ipv4/inet_diag.c | 4 +-- net/ipv4/ip_output.c | 8 ++--- net/ipv4/ip_sockglue.c | 2 +- net/ipv4/raw.c | 2 +- net/ipv4/route.c | 4 +-- net/ipv4/tcp_ipv4.c | 4 +-- net/ipv6/ping.c | 2 +- net/ipv6/raw.c | 6 ++-- net/ipv6/route.c | 7 ++-- net/ipv6/tcp_ipv6.c | 9 ++--- net/ipv6/udp.c | 4 +-- net/l2tp/l2tp_ip6.c | 2 +- net/mptcp/sockopt.c | 2 +- net/netfilter/nft_socket.c | 2 +- net/netfilter/xt_socket.c | 4 +-- net/packet/af_packet.c | 12 +++---- net/sched/em_meta.c | 4 +-- net/smc/af_smc.c | 2 +- net/unix/af_unix.c | 2 +- net/xdp/xsk.c | 2 +- net/xfrm/xfrm_policy.c | 2 +- 27 files changed, 94 insertions(+), 80 deletions(-)
Signature · 1 line
2.41.0.585.gd2178a4bd4-goog
PATCH 1/11 net: annotate data-races around sk->sk_reserved_mem
sk_getsockopt() runs locklessly. This means sk->sk_reserved_mem can be read while other threads are changing its value.
Add missing annotations where they are needed.
--- net/core/sock.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
net/core/sock.c +4 -3
@@ -1007,7 +1007,7 @@ static void sock_release_reserved_memory(struct sock *sk, int bytes) bytes =round_down (bytes,PAGE_SIZE );WARN_ON (bytes > sk->sk_reserved_mem); - sk->sk_reserved_mem -= bytes; +WRITE_ONCE (sk->sk_reserved_mem, sk->sk_reserved_mem - bytes);sk_mem_reclaim (sk); }
@@ -1044,7 +1044,8 @@ static int sock_reserve_memory(struct sock *sk, int bytes) } sk->sk_forward_alloc += pages <<PAGE_SHIFT ; - sk->sk_reserved_mem += pages <<PAGE_SHIFT ; +WRITE_ONCE (sk->sk_reserved_mem, + sk->sk_reserved_mem + (pages <<PAGE_SHIFT ));return 0 ; }
@@ -1973,7 +1974,7 @@ int sk_getsockopt(struct sock *sk, int level, int optname,break ;case SO_RESERVE_MEM : - v.val = sk->sk_reserved_mem; + v.val =READ_ONCE (sk->sk_reserved_mem);break ;case SO_TXREHASH :
Signature · 1 line
2.41.0.585.gd2178a4bd4-goog
PATCH 2/11 net: annotate data-race around sk->sk_txrehash
In reply to Eric Dumazetsk_getsockopt() runs locklessly. This means sk->sk_txrehash can be read while other threads are changing its value.
Other locations were handled in commit cb6cd2cec799 ("tcp: Change SYN ACK retransmit behaviour to account for rehash")
--- net/core/sock.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
net/core/sock.c +5 -2
@@ -1534,7 +1534,9 @@ int sk_setsockopt(struct sock *sk, int level, int optname, }if ((u8 )val ==SOCK_TXREHASH_DEFAULT ) val =READ_ONCE (sock_net (sk)->core.sysctl_txrehash); -/* Paired with READ_ONCE() in tcp_rtx_synack() */ +/* Paired with READ_ONCE() in tcp_rtx_synack() +* and sk_getsockopt(). +*/ WRITE_ONCE (sk->sk_txrehash, (u8 )val);break ;
@@ -1978,7 +1980,8 @@ int sk_getsockopt(struct sock *sk, int level, int optname,break ;case SO_TXREHASH : - v.val = sk->sk_txrehash; +/* Paired with WRITE_ONCE() in sk_setsockopt() */ + v.val =READ_ONCE (sk->sk_txrehash);break ;default :
Signature · 1 line
2.41.0.585.gd2178a4bd4-goog
PATCH 3/11 net: annotate data-races around sk->sk_max_pacing_rate
In reply to Eric Dumazetsk_getsockopt() runs locklessly. This means sk->sk_max_pacing_rate can be read while other threads are changing its value.
--- net/core/sock.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
net/core/sock.c +6 -3
@@ -1439,7 +1439,8 @@ int sk_setsockopt(struct sock *sk, int level, int optname,cmpxchg (&sk->sk_pacing_status,SK_PACING_NONE ,SK_PACING_NEEDED ); - sk->sk_max_pacing_rate = ulval; +/* Pairs with READ_ONCE() from sk_getsockopt() */ +WRITE_ONCE (sk->sk_max_pacing_rate, ulval); sk->sk_pacing_rate =min (sk->sk_pacing_rate, ulval);break ; }
@@ -1903,12 +1904,14 @@ int sk_getsockopt(struct sock *sk, int level, int optname,#endif case SO_MAX_PACING_RATE : +/* The READ_ONCE() pair with the WRITE_ONCE() in sk_setsockopt() */ if (sizeof (v.ulval) !=sizeof (v.val) && len >=sizeof (v.ulval)) { lv =sizeof (v.ulval); - v.ulval = sk->sk_max_pacing_rate; + v.ulval =READ_ONCE (sk->sk_max_pacing_rate); }else {/* 32bit version */ - v.val = min_t(unsigned long, sk->sk_max_pacing_rate, ~0U); + v.val = min_t(unsigned long, ~0U, +READ_ONCE (sk->sk_max_pacing_rate)); }break ;
Signature · 1 line
2.41.0.585.gd2178a4bd4-goog
PATCH 4/11 net: add missing READ_ONCE(sk->sk_rcvlowat) annotation
In reply to Eric DumazetIn a prior commit, I forgot to change sk_getsockopt() when reading sk->sk_rcvlowat locklessly.
--- net/core/sock.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
net/core/sock.c +1 -1
@@ -1730,7 +1730,7 @@ int sk_getsockopt(struct sock *sk, int level, int optname,break ;case SO_RCVLOWAT : - v.val = sk->sk_rcvlowat; + v.val =READ_ONCE (sk->sk_rcvlowat);break ;case SO_SNDLOWAT :
Signature · 1 line
2.41.0.585.gd2178a4bd4-goog
PATCH 5/11 net: annotate data-races around sk->sk_{rcv|snd}timeo
In reply to Eric Dumazetsk_getsockopt() runs without locks, we must add annotations to sk->sk_rcvtimeo and sk->sk_sndtimeo.
In the future we might allow fetching these fields before we lock the socket in TCP fast path.
--- net/core/sock.c | 24 ++++++++++++++---------- net/sched/em_meta.c | 4 ++-- 2 files changed, 16 insertions(+), 12 deletions(-)
net/core/sock.c +14 -10
@@ -429,6 +429,7 @@ static int sock_set_timeout(long *timeo_p, sockptr_t optval, int optlen, {struct __kernel_sock_timeval tv;int err =sock_copy_user_timeval (&tv, optval, optlen, old_timeval); +long val;if (err)return err;
@@ -439,7 +440,7 @@ static int sock_set_timeout(long *timeo_p, sockptr_t optval, int optlen,if (tv.tv_sec <0 ) {static int warned__read_mostly ; - *timeo_p =0 ; +WRITE_ONCE (*timeo_p,0 );if (warned <10 &&net_ratelimit ()) { warned++; pr_info("%s: `%s' (pid %d) tries to set negative timeout\n",
@@ -447,11 +448,12 @@ static int sock_set_timeout(long *timeo_p, sockptr_t optval, int optlen, }return 0 ; } - *timeo_p =MAX_SCHEDULE_TIMEOUT ; -if (tv.tv_sec ==0 && tv.tv_usec ==0 ) -return 0 ; -if (tv.tv_sec < (MAX_SCHEDULE_TIMEOUT /HZ -1 )) - *timeo_p = tv.tv_sec *HZ +DIV_ROUND_UP ((unsigned long )tv.tv_usec,USEC_PER_SEC /HZ ); + val =MAX_SCHEDULE_TIMEOUT ; +if ((tv.tv_sec || tv.tv_usec) && + (tv.tv_sec < (MAX_SCHEDULE_TIMEOUT /HZ -1 ))) + val = tv.tv_sec *HZ +DIV_ROUND_UP ((unsigned long )tv.tv_usec, +USEC_PER_SEC /HZ ); +WRITE_ONCE (*timeo_p, val);return 0 ; }
@@ -813,9 +815,9 @@ void sock_set_sndtimeo(struct sock *sk, s64 secs) {lock_sock (sk);if (secs && secs <MAX_SCHEDULE_TIMEOUT /HZ -1 ) - sk->sk_sndtimeo = secs *HZ ; +WRITE_ONCE (sk->sk_sndtimeo, secs *HZ );else - sk->sk_sndtimeo =MAX_SCHEDULE_TIMEOUT ; +WRITE_ONCE (sk->sk_sndtimeo,MAX_SCHEDULE_TIMEOUT );release_sock (sk); }EXPORT_SYMBOL (sock_set_sndtimeo);
@@ -1721,12 +1723,14 @@ int sk_getsockopt(struct sock *sk, int level, int optname,case SO_RCVTIMEO_OLD :case SO_RCVTIMEO_NEW : - lv =sock_get_timeout (sk->sk_rcvtimeo, &v,SO_RCVTIMEO_OLD == optname); + lv =sock_get_timeout (READ_ONCE (sk->sk_rcvtimeo), &v, +SO_RCVTIMEO_OLD == optname);break ;case SO_SNDTIMEO_OLD :case SO_SNDTIMEO_NEW : - lv =sock_get_timeout (sk->sk_sndtimeo, &v,SO_SNDTIMEO_OLD == optname); + lv =sock_get_timeout (READ_ONCE (sk->sk_sndtimeo), &v, +SO_SNDTIMEO_OLD == optname);break ;case SO_RCVLOWAT :
net/sched/em_meta.c +2 -2
@@ -568,7 +568,7 @@ META_COLLECTOR(int_sk_rcvtimeo) *err =-1 ;return ; } - dst->value = sk->sk_rcvtimeo /HZ ; + dst->value =READ_ONCE (sk->sk_rcvtimeo) /HZ ; }META_COLLECTOR (int_sk_sndtimeo)
@@ -579,7 +579,7 @@ META_COLLECTOR(int_sk_sndtimeo) *err =-1 ;return ; } - dst->value = sk->sk_sndtimeo /HZ ; + dst->value =READ_ONCE (sk->sk_sndtimeo) /HZ ; }META_COLLECTOR (int_sk_sendmsg_off)
Signature · 1 line
2.41.0.585.gd2178a4bd4-goog
PATCH 6/11 net: add missing READ_ONCE(sk->sk_sndbuf) annotation
In reply to Eric DumazetIn a prior commit, I forgot to change sk_getsockopt() when reading sk->sk_sndbuf locklessly.
--- net/core/sock.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
net/core/sock.c +1 -1
@@ -1639,7 +1639,7 @@ int sk_getsockopt(struct sock *sk, int level, int optname,break ;case SO_SNDBUF : - v.val = sk->sk_sndbuf; + v.val =READ_ONCE (sk->sk_sndbuf);break ;case SO_RCVBUF :
Signature · 1 line
2.41.0.585.gd2178a4bd4-goog
PATCH 7/11 net: add missing READ_ONCE(sk->sk_rcvbuf) annotation
In reply to Eric DumazetIn a prior commit, I forgot to change sk_getsockopt() when reading sk->sk_rcvbuf locklessly.
--- net/core/sock.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
net/core/sock.c +1 -1
@@ -1643,7 +1643,7 @@ int sk_getsockopt(struct sock *sk, int level, int optname,break ;case SO_RCVBUF : - v.val = sk->sk_rcvbuf; + v.val =READ_ONCE (sk->sk_rcvbuf);break ;case SO_REUSEADDR :
Signature · 1 line
2.41.0.585.gd2178a4bd4-goog
PATCH 8/11 net: annotate data-races around sk->sk_mark
In reply to Eric Dumazetsk->sk_mark is often read while another thread could change the value.
--- include/net/inet_sock.h | 7 ++++--- include/net/ip.h | 2 +- include/net/route.h | 4 ++-- net/can/raw.c | 2 +- net/core/sock.c | 4 ++-- net/dccp/ipv6.c | 4 ++-- net/ipv4/inet_diag.c | 4 ++-- net/ipv4/ip_output.c | 4 ++-- net/ipv4/route.c | 4 ++-- net/ipv4/tcp_ipv4.c | 2 +- net/ipv6/ping.c | 2 +- net/ipv6/raw.c | 4 ++-- net/ipv6/route.c | 7 ++++--- net/ipv6/tcp_ipv6.c | 6 +++--- net/ipv6/udp.c | 4 ++-- net/l2tp/l2tp_ip6.c | 2 +- net/mptcp/sockopt.c | 2 +- net/netfilter/nft_socket.c | 2 +- net/netfilter/xt_socket.c | 4 ++-- net/packet/af_packet.c | 6 +++--- net/smc/af_smc.c | 2 +- net/xdp/xsk.c | 2 +- net/xfrm/xfrm_policy.c | 2 +- 23 files changed, 42 insertions(+), 40 deletions(-)
include/net/inet_sock.h +4 -3
@@ -107,11 +107,12 @@ static inline struct inet_request_sock *inet_rsk(const struct request_sock *sk)static inline u32 inet_request_mark (const struct sock *sk,struct sk_buff *skb) { -if (!sk->sk_mark && -READ_ONCE (sock_net (sk)->ipv4.sysctl_tcp_fwmark_accept)) +u32 mark =READ_ONCE (sk->sk_mark); + +if (!mark &&READ_ONCE (sock_net (sk)->ipv4.sysctl_tcp_fwmark_accept))return skb->mark; -return sk->sk_mark; +return mark; } static inline int inet_request_bound_dev_if(const struct sock *sk,
include/net/ip.h +1 -1
@@ -93,7 +93,7 @@ static inline void ipcm_init_sk(struct ipcm_cookie *ipcm, {ipcm_init (ipcm); - ipcm->sockc.mark = inet->sk.sk_mark; + ipcm->sockc.mark =READ_ONCE (inet->sk.sk_mark); ipcm->sockc.tsflags = inet->sk.sk_tsflags; ipcm->oif =READ_ONCE (inet->sk.sk_bound_dev_if); ipcm->addr = inet->inet_saddr;
include/net/route.h +2 -2
@@ -168,7 +168,7 @@ static inline struct rtable *ip_route_output_ports(struct net *net, struct flowi __be16 dport, __be16 sport, __u8 proto, __u8 tos, int oif) { - flowi4_init_output(fl4, oif, sk ? sk->sk_mark : 0, tos, + flowi4_init_output(fl4, oif, sk ? READ_ONCE(sk->sk_mark) : 0, tos,RT_SCOPE_UNIVERSE ,proto , sk ? inet_sk_flowi_flags(sk) : 0,daddr ,saddr ,dport ,sport ,sock_net_uid (net ,sk ));
@@ -301,7 +301,7 @@ static inline void ip_route_connect_init(struct flowi4 *fl4, __be32 dst,if (inet_sk (sk)->transparent) flow_flags |=FLOWI_FLAG_ANYSRC ; -flowi4_init_output (fl4, oif, sk->sk_mark,ip_sock_rt_tos (sk), +flowi4_init_output (fl4, oif,READ_ONCE (sk->sk_mark),ip_sock_rt_tos (sk),ip_sock_rt_scope (sk), protocol, flow_flags, dst, src, dport, sport, sk->sk_uid); }
net/can/raw.c +1 -1
@@ -865,7 +865,7 @@ static int raw_sendmsg(struct socket *sock, struct msghdr *msg, size_t size) skb->dev = dev; skb->priority = sk->sk_priority; - skb->mark = sk->sk_mark; + skb->mark =READ_ONCE (sk->sk_mark); skb->tstamp = sockc.transmit_time;skb_setup_tx_timestamp (skb, sockc.tsflags);
net/core/sock.c +2 -2
@@ -990,7 +990,7 @@ EXPORT_SYMBOL(sock_set_rcvbuf);static void __sock_set_mark (struct sock *sk,u32 val) {if (val != sk->sk_mark) { - sk->sk_mark = val; +WRITE_ONCE (sk->sk_mark, val);sk_dst_reset (sk); } }
@@ -1851,7 +1851,7 @@ int sk_getsockopt(struct sock *sk, int level, int optname, optval, optlen, len);case SO_MARK : - v.val = sk->sk_mark; + v.val =READ_ONCE (sk->sk_mark);break ;case SO_RCVMARK :
net/dccp/ipv6.c +2 -2
@@ -238,8 +238,8 @@ static int dccp_v6_send_response(const struct sock *sk, struct request_sock *req opt = ireq->ipv6_opt;if (!opt) opt =rcu_dereference (np->opt); - err =ip6_xmit (sk, skb, &fl6, sk->sk_mark, opt, np->tclass, - sk->sk_priority); + err =ip6_xmit (sk, skb, &fl6,READ_ONCE (sk->sk_mark), opt, + np->tclass, sk->sk_priority);rcu_read_unlock (); err =net_xmit_eval (err); }
net/ipv4/inet_diag.c +2 -2
@@ -150,7 +150,7 @@ int inet_diag_msg_attrs_fill(struct sock *sk, struct sk_buff *skb, }#endif -if (net_admin &&nla_put_u32 (skb,INET_DIAG_MARK , sk->sk_mark)) +if (net_admin &&nla_put_u32 (skb,INET_DIAG_MARK ,READ_ONCE (sk->sk_mark))) goto errout; if (ext & (1 << (INET_DIAG_CLASS_ID - 1)) ||
@@ -799,7 +799,7 @@ int inet_diag_bc_sk(const struct nlattr *bc, struct sock *sk) entry.ifindex = sk->sk_bound_dev_if; entry.userlocks =sk_fullsock (sk) ? sk->sk_userlocks :0 ;if (sk_fullsock (sk)) - entry.mark = sk->sk_mark; + entry.mark =READ_ONCE (sk->sk_mark);else if (sk->sk_state ==TCP_NEW_SYN_RECV ) entry.mark =inet_rsk (inet_reqsk (sk))->ir_mark;else if (sk->sk_state ==TCP_TIME_WAIT )
net/ipv4/ip_output.c +2 -2
@@ -186,7 +186,7 @@ int ip_build_and_send_pkt(struct sk_buff *skb, const struct sock *sk, skb->priority = sk->sk_priority;if (!skb->mark) - skb->mark = sk->sk_mark; + skb->mark =READ_ONCE (sk->sk_mark);/* Send it out. */ return ip_local_out (net, skb->sk, skb);
@@ -529,7 +529,7 @@ int __ip_queue_xmit(struct sock *sk, struct sk_buff *skb, struct flowi *fl,/* TODO : should we use skb->sk here instead of sk ? */ skb->priority = sk->sk_priority; - skb->mark = sk->sk_mark; + skb->mark =READ_ONCE (sk->sk_mark); res =ip_local_out (net, sk, skb);rcu_read_unlock ();
net/ipv4/route.c +2 -2
@@ -518,7 +518,7 @@ static void __build_flow_key(const struct net *net, struct flowi4 *fl4,const struct inet_sock *inet =inet_sk (sk); oif = sk->sk_bound_dev_if; - mark = sk->sk_mark; + mark =READ_ONCE (sk->sk_mark); tos =ip_sock_rt_tos (sk); scope =ip_sock_rt_scope (sk); prot = inet->hdrincl ?IPPROTO_RAW : sk->sk_protocol;
@@ -552,7 +552,7 @@ static void build_sk_flow_key(struct flowi4 *fl4, const struct sock *sk) inet_opt =rcu_dereference (inet->inet_opt);if (inet_opt && inet_opt->opt.srr) daddr = inet_opt->opt.faddr; - flowi4_init_output(fl4, sk->sk_bound_dev_if, sk->sk_mark, + flowi4_init_output(fl4, sk->sk_bound_dev_if, READ_ONCE(sk->sk_mark), ip_sock_rt_tos(sk) & IPTOS_RT_MASK, ip_sock_rt_scope(sk), inet->hdrincl ? IPPROTO_RAW : sk->sk_protocol,
net/ipv4/tcp_ipv4.c +1 -1
@@ -931,7 +931,7 @@ static void tcp_v4_send_ack(const struct sock *sk, ctl_sk =this_cpu_read (ipv4_tcp_sk);sock_net_set (ctl_sk, net); ctl_sk->sk_mark = (sk->sk_state ==TCP_TIME_WAIT ) ? -inet_twsk (sk)->tw_mark : sk->sk_mark; +inet_twsk (sk)->tw_mark :READ_ONCE (sk->sk_mark); ctl_sk->sk_priority = (sk->sk_state ==TCP_TIME_WAIT ) ?inet_twsk (sk)->tw_priority : sk->sk_priority; transmit_time =tcp_transmit_time (sk);
net/ipv6/ping.c +1 -1
@@ -120,7 +120,7 @@ static int ping_v6_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)ipcm6_init_sk (&ipc6, np); ipc6.sockc.tsflags = sk->sk_tsflags; - ipc6.sockc.mark = sk->sk_mark; + ipc6.sockc.mark =READ_ONCE (sk->sk_mark); fl6.flowi6_oif = oif;
net/ipv6/raw.c +2 -2
@@ -774,12 +774,12 @@ static int rawv6_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)*/ memset (&fl6,0 ,sizeof (fl6)); - fl6.flowi6_mark = sk->sk_mark; + fl6.flowi6_mark =READ_ONCE (sk->sk_mark); fl6.flowi6_uid = sk->sk_uid;ipcm6_init (&ipc6); ipc6.sockc.tsflags = sk->sk_tsflags; - ipc6.sockc.mark = sk->sk_mark; + ipc6.sockc.mark = fl6.flowi6_mark;if (sin6) {if (addr_len <SIN6_LEN_RFC2133 )
net/ipv6/route.c +4 -3
@@ -2951,7 +2951,8 @@ void ip6_sk_update_pmtu(struct sk_buff *skb, struct sock *sk, __be32 mtu)if (!oif && skb->dev) oif =l3mdev_master_ifindex (skb->dev); -ip6_update_pmtu (skb,sock_net (sk), mtu, oif, sk->sk_mark, sk->sk_uid); +ip6_update_pmtu (skb,sock_net (sk), mtu, oif,READ_ONCE (sk->sk_mark), + sk->sk_uid); dst =__sk_dst_get (sk); if (!dst || !dst->obsolete ||
@@ -3172,8 +3173,8 @@ void ip6_redirect_no_header(struct sk_buff *skb, struct net *net, int oif)void ip6_sk_redirect (struct sk_buff *skb,struct sock *sk) { -ip6_redirect (skb,sock_net (sk), sk->sk_bound_dev_if, sk->sk_mark, - sk->sk_uid); +ip6_redirect (skb,sock_net (sk), sk->sk_bound_dev_if, +READ_ONCE (sk->sk_mark), sk->sk_uid); }EXPORT_SYMBOL_GPL (ip6_sk_redirect);
net/ipv6/tcp_ipv6.c +3 -3
@@ -564,8 +564,8 @@ static int tcp_v6_send_synack(const struct sock *sk, struct dst_entry *dst, opt = ireq->ipv6_opt;if (!opt) opt =rcu_dereference (np->opt); - err =ip6_xmit (sk, skb, fl6, skb->mark ? : sk->sk_mark, opt, - tclass, sk->sk_priority); + err =ip6_xmit (sk, skb, fl6, skb->mark ? :READ_ONCE (sk->sk_mark), + opt, tclass, sk->sk_priority);rcu_read_unlock (); err =net_xmit_eval (err); }
@@ -939,7 +939,7 @@ static void tcp_v6_send_response(const struct sock *sk, struct sk_buff *skb, u32if (sk->sk_state ==TCP_TIME_WAIT ) mark =inet_twsk (sk)->tw_mark;else - mark = sk->sk_mark; + mark =READ_ONCE (sk->sk_mark);skb_set_delivery_time (buff,tcp_transmit_time (sk), true); }if (txhash) {
net/ipv6/udp.c +2 -2
@@ -624,7 +624,7 @@ int __udp6_lib_err(struct sk_buff *skb, struct inet6_skb_parm *opt,if (type ==NDISC_REDIRECT ) {if (tunnel) {ip6_redirect (skb,sock_net (sk),inet6_iif (skb), - sk->sk_mark, sk->sk_uid); +READ_ONCE (sk->sk_mark), sk->sk_uid); }else {ip6_sk_redirect (skb, sk); }
@@ -1356,7 +1356,7 @@ int udpv6_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)ipcm6_init (&ipc6); ipc6.gso_size =READ_ONCE (up->gso_size); ipc6.sockc.tsflags = sk->sk_tsflags; - ipc6.sockc.mark = sk->sk_mark; + ipc6.sockc.mark =READ_ONCE (sk->sk_mark);/* destination address check */ if (sin6) {
net/l2tp/l2tp_ip6.c +1 -1
@@ -519,7 +519,7 @@ static int l2tp_ip6_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)/* Get and verify the address */ memset (&fl6,0 ,sizeof (fl6)); - fl6.flowi6_mark = sk->sk_mark; + fl6.flowi6_mark =READ_ONCE (sk->sk_mark); fl6.flowi6_uid = sk->sk_uid;ipcm6_init (&ipc6);
net/mptcp/sockopt.c +1 -1
@@ -103,7 +103,7 @@ static void mptcp_sol_socket_sync_intval(struct mptcp_sock *msk, int optname, inbreak ;case SO_MARK :if (READ_ONCE (ssk->sk_mark) != sk->sk_mark) { - ssk->sk_mark = sk->sk_mark; +WRITE_ONCE (ssk->sk_mark, sk->sk_mark);sk_dst_reset (ssk); }break ;
net/netfilter/nft_socket.c +1 -1
@@ -107,7 +107,7 @@ static void nft_socket_eval(const struct nft_expr *expr,break ;case NFT_SOCKET_MARK :if (sk_fullsock (sk)) { - *dest = sk->sk_mark; + *dest =READ_ONCE (sk->sk_mark); }else { regs->verdict.code =NFT_BREAK ;return ;
net/netfilter/xt_socket.c +2 -2
@@ -77,7 +77,7 @@ socket_match(const struct sk_buff *skb, struct xt_action_param *par,if (info->flags &XT_SOCKET_RESTORESKMARK && !wildcard && transparent &&sk_fullsock (sk)) - pskb->mark = sk->sk_mark; + pskb->mark =READ_ONCE (sk->sk_mark);if (sk != skb->sk)sock_gen_put (sk);
@@ -138,7 +138,7 @@ socket_mt6_v1_v2_v3(const struct sk_buff *skb, struct xt_action_param *par)if (info->flags &XT_SOCKET_RESTORESKMARK && !wildcard && transparent &&sk_fullsock (sk)) - pskb->mark = sk->sk_mark; + pskb->mark =READ_ONCE (sk->sk_mark);if (sk != skb->sk)sock_gen_put (sk);
net/packet/af_packet.c +3 -3
@@ -2051,7 +2051,7 @@ static int packet_sendmsg_spkt(struct socket *sock, struct msghdr *msg, skb->protocol = proto; skb->dev = dev; skb->priority = sk->sk_priority; - skb->mark = sk->sk_mark; + skb->mark =READ_ONCE (sk->sk_mark); skb->tstamp = sockc.transmit_time;skb_setup_tx_timestamp (skb, sockc.tsflags);
@@ -2586,7 +2586,7 @@ static int tpacket_fill_skb(struct packet_sock *po, struct sk_buff *skb, skb->protocol = proto; skb->dev = dev; skb->priority = po->sk.sk_priority; - skb->mark = po->sk.sk_mark; + skb->mark =READ_ONCE (po->sk.sk_mark); skb->tstamp = sockc->transmit_time;skb_setup_tx_timestamp (skb, sockc->tsflags);skb_zcopy_set_nouarg (skb, ph.raw);
@@ -2988,7 +2988,7 @@ static int packet_snd(struct socket *sock, struct msghdr *msg, size_t len) goto out_unlock;sockcm_init (&sockc, sk); - sockc.mark = sk->sk_mark; + sockc.mark =READ_ONCE (sk->sk_mark);if (msg->msg_controllen) { err =sock_cmsg_send (sk, msg, &sockc);if (unlikely (err))
net/smc/af_smc.c +1 -1
@@ -445,7 +445,7 @@ static void smc_copy_sock_settings(struct sock *nsk, struct sock *osk, nsk->sk_rcvbuf = osk->sk_rcvbuf; nsk->sk_sndtimeo = osk->sk_sndtimeo; nsk->sk_rcvtimeo = osk->sk_rcvtimeo; - nsk->sk_mark = osk->sk_mark; + nsk->sk_mark =READ_ONCE (osk->sk_mark); nsk->sk_priority = osk->sk_priority; nsk->sk_rcvlowat = osk->sk_rcvlowat; nsk->sk_bound_dev_if = osk->sk_bound_dev_if;
net/xdp/xsk.c +1 -1
@@ -505,7 +505,7 @@ static struct sk_buff *xsk_build_skb(struct xdp_sock *xs, skb->dev = dev; skb->priority = xs->sk.sk_priority; - skb->mark = xs->sk.sk_mark; + skb->mark =READ_ONCE (xs->sk.sk_mark);skb_shinfo (skb)->destructor_arg = (void *)(long )desc->addr; skb->destructor = xsk_destruct_skb;
net/xfrm/xfrm_policy.c +1 -1
@@ -2250,7 +2250,7 @@ static struct xfrm_policy *xfrm_sk_policy_lookup(const struct sock *sk, int dir, match =xfrm_selector_match (&pol->selector, fl, family);if (match) { -if ((sk->sk_mark & pol->mark.m) != pol->mark.v || +if ((READ_ONCE (sk->sk_mark) & pol->mark.m) != pol->mark.v || pol->if_id != if_id) { pol =NULL ; goto out;
Signature · 1 line
2.41.0.585.gd2178a4bd4-goog
Re: [PATCH net 08/11] net: annotate data-races around sk->sk_mark - manual merge
Hello,
Quoted 4 lines · Eric Dumazet
sk->sk_markis often read while another thread could change the value.Fixes 4a19ec5800fc ("[NET]: Introducing socket mark socket option.")Signed-off-by Eric Dumazet <[email protected]>
FYI, we got a small conflict when merging 'net' in 'net-next' in the MPTCP tree due to this patch applied in 'net':
3c5b4d69c358 ("net: annotate data-races around sk->sk_mark")
and this one from 'net-next':
b7f72a30e9ac ("xsk: introduce wrappers and helpers for supporting
multi-buffer in Tx path")
Regarding this conflict, this is a trivial context-based one: I simply took the modifications from the two versions.
Rerere cache is available in [2].
Cheers,
Matt
[1] https://github.com/multipath-tcp/mptcp_net-next/commit/083975145b2e
[2] https://github.com/multipath-tcp/mptcp-upstream-rr-cache/commit/0616
Signature · 2 lines
Tessares | Belgium | Hybrid Access Solutions www.tessares.net
net/xdp/xsk.c +0 -0
@@@ -682,22 -505,11 +682,22 @@@ static struct sk_buff *xsk_build_skb(st skb->dev = dev; skb->priority = xs->sk.sk_priority; - skb->mark = xs->sk.sk_mark; + skb->mark =READ_ONCE (xs->sk.sk_mark); -skb_shinfo (skb)->destructor_arg = (void *)(long )desc->addr; skb->destructor = xsk_destruct_skb; +xsk_set_destructor_arg (skb);return skb; + +free_err: + if (err == -EAGAIN) { +xsk_cq_cancel_locked (xs,1 ); + } else { +xsk_set_destructor_arg (skb); +xsk_drop_skb (skb); +xskq_cons_release (xs->tx); + } + + return ERR_PTR(err); }static int __xsk_generic_xmit (struct sock *sk)
PATCH 9/11 net: add missing data-race annotations around sk->sk_peek_off
In reply to Eric Dumazetsk_getsockopt() runs locklessly, thus we need to annotate the read of sk->sk_peek_off.
While we are at it, add corresponding annotations to sk_set_peek_off() and unix_set_peek_off().
--- net/core/sock.c | 4 ++-- net/unix/af_unix.c | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-)
net/core/sock.c +2 -2
@@ -1870,7 +1870,7 @@ int sk_getsockopt(struct sock *sk, int level, int optname,if (!sock->ops->set_peek_off)return -EOPNOTSUPP ; - v.val = sk->sk_peek_off; + v.val =READ_ONCE (sk->sk_peek_off);break ;case SO_NOFCS : v.val =sock_flag (sk,SOCK_NOFCS );
@@ -3179,7 +3179,7 @@ EXPORT_SYMBOL(__sk_mem_reclaim);int sk_set_peek_off (struct sock *sk,int val) { - sk->sk_peek_off = val; +WRITE_ONCE (sk->sk_peek_off, val);return 0 ; }EXPORT_SYMBOL_GPL (sk_set_peek_off);
net/unix/af_unix.c +1 -1
@@ -790,7 +790,7 @@ static int unix_set_peek_off(struct sock *sk, int val)if (mutex_lock_interruptible (&u->iolock))return -EINTR ; - sk->sk_peek_off = val; +WRITE_ONCE (sk->sk_peek_off, val);mutex_unlock (&u->iolock);return 0 ;
Signature · 1 line
2.41.0.585.gd2178a4bd4-goog
PATCH 10/11 net: add missing data-race annotation for sk_ll_usec
In reply to Eric DumazetIn a prior commit I forgot that sk_getsockopt() reads sk->sk_ll_usec without holding a lock.
--- net/core/sock.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
net/core/sock.c +1 -1
@@ -1900,7 +1900,7 @@ int sk_getsockopt(struct sock *sk, int level, int optname,#ifdef CONFIG_NET_RX_BUSY_POLL case SO_BUSY_POLL : - v.val = sk->sk_ll_usec; + v.val =READ_ONCE (sk->sk_ll_usec);break ;case SO_PREFER_BUSY_POLL : v.val =READ_ONCE (sk->sk_prefer_busy_poll);
Signature · 1 line
2.41.0.585.gd2178a4bd4-goog
PATCH 11/11 net: annotate data-races around sk->sk_priority
In reply to Eric Dumazetsk_getsockopt() runs locklessly. This means sk->sk_priority can be read while other threads are changing its value.
Other reads also happen without socket lock being held.
Add missing annotations where needed.
--- net/core/sock.c | 6 +++--- net/ipv4/ip_output.c | 4 ++-- net/ipv4/ip_sockglue.c | 2 +- net/ipv4/raw.c | 2 +- net/ipv4/tcp_ipv4.c | 2 +- net/ipv6/raw.c | 2 +- net/ipv6/tcp_ipv6.c | 3 ++- net/packet/af_packet.c | 6 +++--- 8 files changed, 14 insertions(+), 13 deletions(-)
net/core/sock.c +3 -3
@@ -806,7 +806,7 @@ EXPORT_SYMBOL(sock_no_linger);void sock_set_priority (struct sock *sk,u32 priority) {lock_sock (sk); - sk->sk_priority = priority; +WRITE_ONCE (sk->sk_priority, priority);release_sock (sk); }EXPORT_SYMBOL (sock_set_priority);
@@ -1216,7 +1216,7 @@ int sk_setsockopt(struct sock *sk, int level, int optname,if ((val >=0 && val <=6 ) ||sockopt_ns_capable (sock_net (sk)->user_ns,CAP_NET_RAW ) ||sockopt_ns_capable (sock_net (sk)->user_ns,CAP_NET_ADMIN )) - sk->sk_priority = val; +WRITE_ONCE (sk->sk_priority, val);else ret = -EPERM ;break ;
@@ -1685,7 +1685,7 @@ int sk_getsockopt(struct sock *sk, int level, int optname,break ;case SO_PRIORITY : - v.val = sk->sk_priority; + v.val =READ_ONCE (sk->sk_priority);break ;case SO_LINGER :
net/ipv4/ip_output.c +2 -2
@@ -184,7 +184,7 @@ int ip_build_and_send_pkt(struct sk_buff *skb, const struct sock *sk,ip_options_build (skb, &opt->opt, daddr, rt); } - skb->priority = sk->sk_priority; + skb->priority =READ_ONCE (sk->sk_priority);if (!skb->mark) skb->mark =READ_ONCE (sk->sk_mark);
@@ -528,7 +528,7 @@ int __ip_queue_xmit(struct sock *sk, struct sk_buff *skb, struct flowi *fl, skb_shinfo(skb)->gso_segs ?: 1);/* TODO : should we use skb->sk here instead of sk ? */ - skb->priority = sk->sk_priority; + skb->priority =READ_ONCE (sk->sk_priority); skb->mark =READ_ONCE (sk->sk_mark); res =ip_local_out (net, sk, skb);
net/ipv4/ip_sockglue.c +1 -1
@@ -592,7 +592,7 @@ void __ip_sock_set_tos(struct sock *sk, int val) }if (inet_sk (sk)->tos != val) {inet_sk (sk)->tos = val; - sk->sk_priority =rt_tos2priority (val); +WRITE_ONCE (sk->sk_priority,rt_tos2priority (val));sk_dst_reset (sk); } }
net/ipv4/raw.c +1 -1
@@ -348,7 +348,7 @@ static int raw_send_hdrinc(struct sock *sk, struct flowi4 *fl4, goto error;skb_reserve (skb, hlen); - skb->priority = sk->sk_priority; + skb->priority =READ_ONCE (sk->sk_priority); skb->mark = sockc->mark; skb->tstamp = sockc->transmit_time;skb_dst_set (skb, &rt->dst);
net/ipv4/tcp_ipv4.c +1 -1
@@ -933,7 +933,7 @@ static void tcp_v4_send_ack(const struct sock *sk, ctl_sk->sk_mark = (sk->sk_state ==TCP_TIME_WAIT ) ?inet_twsk (sk)->tw_mark :READ_ONCE (sk->sk_mark); ctl_sk->sk_priority = (sk->sk_state ==TCP_TIME_WAIT ) ? -inet_twsk (sk)->tw_priority : sk->sk_priority; +inet_twsk (sk)->tw_priority :READ_ONCE (sk->sk_priority); transmit_time =tcp_transmit_time (sk); ip_send_unicast_reply(ctl_sk, skb, &TCP_SKB_CB(skb)->header.h4.opt,
net/ipv6/raw.c +1 -1
@@ -614,7 +614,7 @@ static int rawv6_send_hdrinc(struct sock *sk, struct msghdr *msg, int length,skb_reserve (skb, hlen); skb->protocol =htons (ETH_P_IPV6 ); - skb->priority = sk->sk_priority; + skb->priority =READ_ONCE (sk->sk_priority); skb->mark = sockc->mark; skb->tstamp = sockc->transmit_time;
net/ipv6/tcp_ipv6.c +2 -1
@@ -1128,7 +1128,8 @@ static void tcp_v6_reqsk_send_ack(const struct sock *sk, struct sk_buff *skb,tcp_time_stamp_raw () +tcp_rsk (req)->ts_off,READ_ONCE (req->ts_recent), sk->sk_bound_dev_if,tcp_v6_md5_do_lookup (sk, &ipv6_hdr (skb)->saddr, l3index), -ipv6_get_dsfield (ipv6_hdr (skb)),0 , sk->sk_priority, +ipv6_get_dsfield (ipv6_hdr (skb)),0 , +READ_ONCE (sk->sk_priority), READ_ONCE(tcp_rsk(req)->txhash)); }
net/packet/af_packet.c +3 -3
@@ -2050,7 +2050,7 @@ static int packet_sendmsg_spkt(struct socket *sock, struct msghdr *msg, skb->protocol = proto; skb->dev = dev; - skb->priority = sk->sk_priority; + skb->priority =READ_ONCE (sk->sk_priority); skb->mark =READ_ONCE (sk->sk_mark); skb->tstamp = sockc.transmit_time;
@@ -2585,7 +2585,7 @@ static int tpacket_fill_skb(struct packet_sock *po, struct sk_buff *skb, skb->protocol = proto; skb->dev = dev; - skb->priority = po->sk.sk_priority; + skb->priority =READ_ONCE (po->sk.sk_priority); skb->mark =READ_ONCE (po->sk.sk_mark); skb->tstamp = sockc->transmit_time;skb_setup_tx_timestamp (skb, sockc->tsflags);
@@ -3061,7 +3061,7 @@ static int packet_snd(struct socket *sock, struct msghdr *msg, size_t len) skb->protocol = proto; skb->dev = dev; - skb->priority = sk->sk_priority; + skb->priority =READ_ONCE (sk->sk_priority); skb->mark = sockc.mark; skb->tstamp = sockc.transmit_time;
Signature · 1 line
2.41.0.585.gd2178a4bd4-goog
This series was applied to netdev/net.git (main) by David S. Miller <[email protected]>:
Hello:
This series was applied to netdev/net.git (main)
by David S. Miller <[email protected]>:
Quoted 19 lines · On Fri, 28 Jul 2023 15:03:07 +0000 you wrote:
This series was inspired by a syzbot/KCSAN report.
This will later also permit some optimizations, like not having to lock the socket while reading/writing some of its fields.
Eric Dumazet (11):
net: annotate data-races aroundsk->sk_reserved_mem
net: annotate data-race aroundsk->sk_txrehash
net: annotate data-races aroundsk->sk_max_pacing_rate
net: add missingREAD_ONCE(sk->sk_rcvlowat)annotation
net: annotate data-races around sk->sk_{rcv|snd}timeo
net: add missingREAD_ONCE(sk->sk_sndbuf)annotation
net: add missingREAD_ONCE(sk->sk_rcvbuf)annotation
net: annotate data-races aroundsk->sk_mark
net: add missing data-race annotations aroundsk->sk_peek_off
net: add missing data-race annotation forsk_ll_usec
net: annotate data-races aroundsk->sk_priority
Quoted 1 line
[...]
Here is the summary with links:
- [net,01/11] net: annotate data-races around sk->sk_reserved_mem
https://git.kernel.org/netdev/net/c/fe11fdcb4207 - [net,02/11] net: annotate data-race around sk->sk_txrehash https://git.kernel.org/netdev/net/c/c76a0328899b - [net,03/11] net: annotate data-races around sk->sk_max_pacing_rate https://git.kernel.org/netdev/net/c/ea7f45ef77b3 - [net,04/11] net: add missing READ_ONCE(sk->sk_rcvlowat) annotation https://git.kernel.org/netdev/net/c/e6d12bdb435d - [net,05/11] net: annotate data-races around sk->sk_{rcv|snd}timeo https://git.kernel.org/netdev/net/c/285975dd6742 - [net,06/11] net: add missing READ_ONCE(sk->sk_sndbuf) annotation https://git.kernel.org/netdev/net/c/74bc084327c6 - [net,07/11] net: add missing READ_ONCE(sk->sk_rcvbuf) annotation https://git.kernel.org/netdev/net/c/b4b553253091 - [net,08/11] net: annotate data-races around sk->sk_mark https://git.kernel.org/netdev/net/c/3c5b4d69c358 - [net,09/11] net: add missing data-race annotations around sk->sk_peek_off https://git.kernel.org/netdev/net/c/11695c6e966b - [net,10/11] net: add missing data-race annotation for sk_ll_usec https://git.kernel.org/netdev/net/c/e5f0d2dd3c2f - [net,11/11] net: annotate data-races around sk->sk_priority https://git.kernel.org/netdev/net/c/8bf43be799d4
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html