dccp/tcp: Fix bhash2 issues related to WARN_ON() in inet_csk_get_port().
Subsystems (5)
v3 cover dccp/tcp: Fix bhash2 issues related to WARN_ON() in inet_csk_get_port().
syzkaller was hitting a WARN_ON() in inet_csk_get_port() in the 4th patch, which was because we forgot to fix up bhash2 bucket when connect() for a socket bound to a wildcard address fails in __inet_stream_connect().
There was a similar report [0], but its repro does not fire the WARN_ON() due to inconsistent error handling.
When connect() for a socket bound to a wildcard address fails, saddr may or may not be reset depending on where the failure happens. When we fail in __inet_stream_connect(), sk->sk_prot->disconnect() resets saddr. OTOH, in (dccp|tcp)_v[46]_connect(), if we fail after inet_hash6?_connect(), we forget to reset saddr.
We fix this inconsistent error handling in the 1st patch, and then we'll fix the bhash2 WARN_ON() issue.
Note that there is still an issue in that we reset saddr without checking if there are conflicting sockets in bhash and bhash2, but this should be another series.
See [1][2] for the previous discussion.
[0]: https://lore.kernel.org/netdev/[email protected]/
[1]: https://lore.kernel.org/netdev/[email protected]/
[2]: https://lore.kernel.org/netdev/[email protected]/
[3]: https://lore.kernel.org/netdev/[email protected]/T/#m00aafedb29ff0b55d5e67aef0252ef1baaf4b6ee
Changes:
v3:
* Patch 3
* Update saddr under the bhash's lock
* Correct Fixes tag
* Change #ifdef in inet_update_saddr() along the recent discussion [3]
v2: https://lore.kernel.org/netdev/[email protected]/
* Add patch 2-4
v1: [2]
Kuniyuki Iwashima (4):
dccp/tcp: Reset saddr on failure after inet6?_hash_connect(). dccp/tcp: Remove NULL check for prev_saddr in
inet_bhash2_update_saddr().
dccp/tcp: Update saddr under bhash's lock. dccp/tcp: Fixup bhash2 bucket when connect() fails.
include/net/inet_hashtables.h | 3 +- net/dccp/ipv4.c | 23 ++------- net/dccp/ipv6.c | 24 ++------- net/dccp/proto.c | 3 +- net/ipv4/af_inet.c | 11 +---- net/ipv4/inet_hashtables.c | 91 +++++++++++++++++++++++++++++------ net/ipv4/tcp.c | 3 +- net/ipv4/tcp_ipv4.c | 21 ++------ net/ipv6/tcp_ipv6.c | 20 ++------ 9 files changed, 101 insertions(+), 98 deletions(-)
Signature · 1 line
2.30.2
PATCH v3 1/4 dccp/tcp: Reset saddr on failure after inet6?_hash_connect().
When connect() is called on a socket bound to the wildcard address, we change the socket's saddr to a local address. If the socket fails to connect() to the destination, we have to reset the saddr.
However, when an error occurs after inet_hash6?_connect() in (dccp|tcp)_v[46]_conect(), we forget to reset saddr and leave the socket bound to the address.
From the user's point of view, whether saddr is reset or not varies with errno. Let's fix this inconsistent behaviour.
Note that after this patch, the repro [0] will trigger the WARN_ON() in inet_csk_get_port() again, but this patch is not buggy and rather fixes a bug papering over the bhash2's bug for which we need another fix.
For the record, the repro causes -EADDRNOTAVAIL in inet_hash6_connect() by this sequence:
s1 = socket()
s1.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1) s1.bind(('127.0.0.1', 10000)) s1.sendto(b'hello', MSG_FASTOPEN, (('127.0.0.1', 10000)))
# or s1.connect(('127.0.0.1', 10000)) s2 = socket()
s2.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1) s2.bind(('0.0.0.0', 10000)) s2.connect(('127.0.0.1', 10000)) # -EADDRNOTAVAIL
s2.listen(32) # WARN_ON(inet_csk(sk)->icsk_bind2_hash != tb2);
[0]: https://syzkaller.appspot.com/bug?extid5d756bbd1f8b5c8f09
--- net/dccp/ipv4.c | 2 ++ net/dccp/ipv6.c | 2 ++ net/ipv4/tcp_ipv4.c | 2 ++ net/ipv6/tcp_ipv6.c | 2 ++ 4 files changed, 8 insertions(+)
net/dccp/ipv4.c +2 -0
@@ -157,6 +157,8 @@ int dccp_v4_connect(struct sock *sk, struct sockaddr *uaddr, int addr_len)* This unhashes the socket and releases the local port, if necessary. */ dccp_set_state (sk,DCCP_CLOSED ); +if (!(sk->sk_userlocks &SOCK_BINDADDR_LOCK )) +inet_reset_saddr (sk);ip_rt_put (rt); sk->sk_route_caps =0 ; inet->inet_dport =0 ;
net/dccp/ipv6.c +2 -0
@@ -985,6 +985,8 @@ static int dccp_v6_connect(struct sock *sk, struct sockaddr *uaddr, late_failure:dccp_set_state (sk,DCCP_CLOSED ); +if (!(sk->sk_userlocks &SOCK_BINDADDR_LOCK )) +inet_reset_saddr (sk);__sk_dst_reset (sk); failure: inet->inet_dport =0 ;
net/ipv4/tcp_ipv4.c +2 -0
@@ -343,6 +343,8 @@ int tcp_v4_connect(struct sock *sk, struct sockaddr *uaddr, int addr_len)* if necessary. */ tcp_set_state (sk,TCP_CLOSE ); +if (!(sk->sk_userlocks &SOCK_BINDADDR_LOCK )) +inet_reset_saddr (sk);ip_rt_put (rt); sk->sk_route_caps =0 ; inet->inet_dport =0 ;
net/ipv6/tcp_ipv6.c +2 -0
@@ -359,6 +359,8 @@ static int tcp_v6_connect(struct sock *sk, struct sockaddr *uaddr, late_failure:tcp_set_state (sk,TCP_CLOSE ); +if (!(sk->sk_userlocks &SOCK_BINDADDR_LOCK )) +inet_reset_saddr (sk); failure: inet->inet_dport =0 ; sk->sk_route_caps =0 ;
Signature · 1 line
2.30.2
PATCH v3 2/4 dccp/tcp: Remove NULL check for prev_saddr in inet_bhash2_update_saddr().
In reply to Kuniyuki IwashimaWhen we call inet_bhash2_update_saddr(), prev_saddr is always non-NULL. Let's remove the unnecessary test.
--- net/ipv4/inet_hashtables.c | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-)
net/ipv4/inet_hashtables.c +4 -7
@@ -877,13 +877,10 @@ int inet_bhash2_update_saddr(struct inet_bind_hashbucket *prev_saddr, struct soc head2 =inet_bhashfn_portaddr (hinfo, sk, net, port); -if (prev_saddr) { -spin_lock_bh (&prev_saddr->lock); -__sk_del_bind2_node (sk); -inet_bind2_bucket_destroy (hinfo->bind2_bucket_cachep, -inet_csk (sk)->icsk_bind2_hash); -spin_unlock_bh (&prev_saddr->lock); - } +spin_lock_bh (&prev_saddr->lock); +__sk_del_bind2_node (sk); +inet_bind2_bucket_destroy (hinfo->bind2_bucket_cachep,inet_csk (sk)->icsk_bind2_hash); +spin_unlock_bh (&prev_saddr->lock);spin_lock_bh (&head2->lock); tb2 =inet_bind2_bucket_find (head2, net, port, l3mdev, sk);
Signature · 1 line
2.30.2
PATCH v3 3/4 dccp/tcp: Update saddr under bhash's lock.
In reply to Kuniyuki IwashimaWhen we call connect() for a socket bound to a wildcard address, we update saddr locklessly. However, it could result in a data race; another thread iterating over bhash might see a corrupted address.
Let's update saddr under the bhash bucket's lock.
--- include/net/inet_hashtables.h | 2 +- net/dccp/ipv4.c | 22 +++----------- net/dccp/ipv6.c | 23 +++------------ net/ipv4/af_inet.c | 11 +------ net/ipv4/inet_hashtables.c | 55 +++++++++++++++++++++++++++++------ net/ipv4/tcp_ipv4.c | 20 +++---------- net/ipv6/tcp_ipv6.c | 19 ++---------- 7 files changed, 63 insertions(+), 89 deletions(-)
include/net/inet_hashtables.h +1 -1
@@ -281,7 +281,7 @@ inet_bhash2_addr_any_hashbucket(const struct sock *sk, const struct net *net, in* sk_v6_rcv_saddr (ipv6) changes after it has been binded. The socket's * rcv_saddr field should already have been updated when this is called. */ -int inet_bhash2_update_saddr (struct inet_bind_hashbucket *prev_saddr,struct sock *sk); +int inet_bhash2_update_saddr (struct sock *sk,void *saddr,int family);void inet_bind_hash (struct sock *sk,struct inet_bind_bucket *tb,struct inet_bind2_bucket *tb2,unsigned short port);
net/dccp/ipv4.c +4 -17
@@ -45,11 +45,10 @@ static unsigned int dccp_v4_pernet_id __read_mostly;int dccp_v4_connect (struct sock *sk,struct sockaddr *uaddr,int addr_len) {const struct sockaddr_in *usin = (struct sockaddr_in *)uaddr; -struct inet_bind_hashbucket *prev_addr_hashbucket =NULL ; -__be32 daddr, nexthop, prev_sk_rcv_saddr;struct inet_sock *inet =inet_sk (sk);struct dccp_sock *dp =dccp_sk (sk);__be16 orig_sport, orig_dport; +__be32 daddr, nexthop;struct flowi4 *fl4;struct rtable *rt;int err;
@@ -91,26 +90,13 @@ int dccp_v4_connect(struct sock *sk, struct sockaddr *uaddr, int addr_len) daddr = fl4->daddr;if (inet->inet_saddr =0 ) { -if (inet_csk (sk)->icsk_bind2_hash) { - prev_addr_hashbucket -inet_bhashfn_portaddr (&dccp_hashinfo, sk, -sock_net (sk), - inet->inet_num); - prev_sk_rcv_saddr = sk->sk_rcv_saddr; - } - inet->inet_saddr = fl4->saddr; - } - -sk_rcv_saddr_set (sk, inet->inet_saddr); - -if (prev_addr_hashbucket) { - err =inet_bhash2_update_saddr (prev_addr_hashbucket, sk); + err =inet_bhash2_update_saddr (sk, &fl4->saddr,AF_INET );if (err) { - inet->inet_saddr =0 ; -sk_rcv_saddr_set (sk, prev_sk_rcv_saddr);ip_rt_put (rt);return err; } + }else { +sk_rcv_saddr_set (sk, inet->inet_saddr); } inet->inet_dport = usin->sin_port;
net/dccp/ipv6.c +4 -19
@@ -934,26 +934,11 @@ static int dccp_v6_connect(struct sock *sk, struct sockaddr *uaddr, }if (saddr =NULL ) { -struct inet_bind_hashbucket *prev_addr_hashbucket =NULL ; -struct in6_addr prev_v6_rcv_saddr; - -if (icsk->icsk_bind2_hash) { - prev_addr_hashbucket =inet_bhashfn_portaddr (&dccp_hashinfo, - sk,sock_net (sk), - inet->inet_num); - prev_v6_rcv_saddr = sk->sk_v6_rcv_saddr; - } - saddr = &fl6.saddr; - sk->sk_v6_rcv_saddr = *saddr; - -if (prev_addr_hashbucket) { - err =inet_bhash2_update_saddr (prev_addr_hashbucket, sk); -if (err) { - sk->sk_v6_rcv_saddr = prev_v6_rcv_saddr; - goto failure; - } - } + + err =inet_bhash2_update_saddr (sk, saddr,AF_INET6 ); +if (err) + goto failure; }/* set the source address */
net/ipv4/af_inet.c +1 -9
@@ -1230,7 +1230,6 @@ EXPORT_SYMBOL(inet_unregister_protosw);static int inet_sk_reselect_saddr (struct sock *sk) { -struct inet_bind_hashbucket *prev_addr_hashbucket;struct inet_sock *inet =inet_sk (sk);__be32 old_saddr = inet->inet_saddr;__be32 daddr = inet->inet_daddr;
@@ -1260,16 +1259,8 @@ static int inet_sk_reselect_saddr(struct sock *sk)return 0 ; } - prev_addr_hashbucket -inet_bhashfn_portaddr (tcp_or_dccp_get_hashinfo (sk), sk, -sock_net (sk), inet->inet_num); - - inet->inet_saddr = inet->inet_rcv_saddr = new_saddr; - - err =inet_bhash2_update_saddr (prev_addr_hashbucket, sk); + err =inet_bhash2_update_saddr (sk, &new_saddr,AF_INET );if (err) { - inet->inet_saddr = old_saddr; - inet->inet_rcv_saddr = old_saddr;ip_rt_put (rt);return err; }
net/ipv4/inet_hashtables.c +46 -9
89 lines not on this page · raw message
net/ipv4/tcp_ipv4.c +4 -16
@@ -199,15 +199,14 @@ static int tcp_v4_pre_connect(struct sock *sk, struct sockaddr *uaddr,/* This will initiate an outgoing connection. */ int tcp_v4_connect (struct sock *sk,struct sockaddr *uaddr,int addr_len) { -struct inet_bind_hashbucket *prev_addr_hashbucket =NULL ;struct sockaddr_in *usin = (struct sockaddr_in *)uaddr;struct inet_timewait_death_row *tcp_death_row; -__be32 daddr, nexthop, prev_sk_rcv_saddr;struct inet_sock *inet =inet_sk (sk);struct tcp_sock *tp =tcp_sk (sk);struct ip_options_rcu *inet_opt;struct net *net =sock_net (sk);__be16 orig_sport, orig_dport; +__be32 daddr, nexthop;struct flowi4 *fl4;struct rtable *rt;int err;
@@ -251,24 +250,13 @@ int tcp_v4_connect(struct sock *sk, struct sockaddr *uaddr, int addr_len) tcp_death_row = &sock_net (sk)->ipv4.tcp_death_row;if (!inet->inet_saddr) { -if (inet_csk (sk)->icsk_bind2_hash) { - prev_addr_hashbucket =inet_bhashfn_portaddr (tcp_death_row->hashinfo, - sk, net, inet->inet_num); - prev_sk_rcv_saddr = sk->sk_rcv_saddr; - } - inet->inet_saddr = fl4->saddr; - } - -sk_rcv_saddr_set (sk, inet->inet_saddr); - -if (prev_addr_hashbucket) { - err =inet_bhash2_update_saddr (prev_addr_hashbucket, sk); + err =inet_bhash2_update_saddr (sk, &fl4->saddr,AF_INET );if (err) { - inet->inet_saddr =0 ; -sk_rcv_saddr_set (sk, prev_sk_rcv_saddr);ip_rt_put (rt);return err; } + }else { +sk_rcv_saddr_set (sk, inet->inet_saddr); }if (tp->rx_opt.ts_recent_stamp && inet->inet_daddr != daddr) {
net/ipv6/tcp_ipv6.c +3 -16
@@ -292,24 +292,11 @@ static int tcp_v6_connect(struct sock *sk, struct sockaddr *uaddr, tcp_death_row = &sock_net (sk)->ipv4.tcp_death_row;if (!saddr) { -struct inet_bind_hashbucket *prev_addr_hashbucket =NULL ; -struct in6_addr prev_v6_rcv_saddr; - -if (icsk->icsk_bind2_hash) { - prev_addr_hashbucket =inet_bhashfn_portaddr (tcp_death_row->hashinfo, - sk, net, inet->inet_num); - prev_v6_rcv_saddr = sk->sk_v6_rcv_saddr; - } saddr = &fl6.saddr; - sk->sk_v6_rcv_saddr = *saddr; -if (prev_addr_hashbucket) { - err =inet_bhash2_update_saddr (prev_addr_hashbucket, sk); -if (err) { - sk->sk_v6_rcv_saddr = prev_v6_rcv_saddr; - goto failure; - } - } + err =inet_bhash2_update_saddr (sk, saddr,AF_INET6 ); +if (err) + goto failure; }/* set the source address */
Signature · 1 line
2.30.2
Re: [PATCH v3 net 3/4] dccp/tcp: Update saddr under bhash's lock.
Quoted 6 lines · Kuniyuki Iwashima
When we call
connect()for a socket bound to a wildcard address, we update saddr locklessly. However, it could result in a data race; another thread iterating over bhash might see a corrupted address.Let's update saddr under the bhash bucket's lock.
Thanks for the quick turnaround!
Quoted 166 lines
Fixes 3df80d9320bc ("[DCCP]: Introduce DCCPv6")Fixes 7c657876b63c ("[DCCP]: Initial implementation")Fixes 1da177e4c3f4 ("Linux-2.6.12-rc2")Signed-off-by Kuniyuki Iwashima <[email protected]>--- include/net/inet_hashtables.h | 2 +- net/dccp/ipv4.c | 22 +++----------- net/dccp/ipv6.c | 23 +++------------ net/ipv4/af_inet.c | 11 +------ net/ipv4/inet_hashtables.c | 55 +++++++++++++++++++++++++++++------ net/ipv4/tcp_ipv4.c | 20 +++---------- net/ipv6/tcp_ipv6.c | 19 ++---------- 7 files changed, 63 insertions(+), 89 deletions(-)include/net/inet_hashtables.h +1
-1@@ -281,7 +281,7 @@ inet_bhash2_addr_any_hashbucket(const struct sock *sk, const struct net *net, in* sk_v6_rcv_saddr (ipv6) changes after it has been binded. The socket's * rcv_saddr field should already have been updated when this is called. */ -int inet_bhash2_update_saddr (struct inet_bind_hashbucket *prev_saddr,struct sock *sk); +int inet_bhash2_update_saddr (struct sock *sk,void *saddr,int family);void inet_bind_hash (struct sock *sk,struct inet_bind_bucket *tb,struct inet_bind2_bucket *tb2,unsigned short port);net/dccp/ipv4.c +4
-17@@ -45,11 +45,10 @@ static unsigned int dccp_v4_pernet_id __read_mostly;int dccp_v4_connect (struct sock *sk,struct sockaddr *uaddr,int addr_len) {const struct sockaddr_in *usin = (struct sockaddr_in *)uaddr; -struct inet_bind_hashbucket *prev_addr_hashbucket =NULL ; -__be32 daddr, nexthop, prev_sk_rcv_saddr;struct inet_sock *inet =inet_sk (sk);struct dccp_sock *dp =dccp_sk (sk);__be16 orig_sport, orig_dport; +__be32 daddr, nexthop;struct flowi4 *fl4;struct rtable *rt;int err;@@ -91,26 +90,13 @@ int dccp_v4_connect(struct sock *sk, struct sockaddr *uaddr, int addr_len) daddr = fl4->daddr;if (inet->inet_saddr =0 ) { -if (inet_csk (sk)->icsk_bind2_hash) { - prev_addr_hashbucket > -inet_bhashfn_portaddr (&dccp_hashinfo, sk, -sock_net (sk), - inet->inet_num); - prev_sk_rcv_saddr = sk->sk_rcv_saddr; - } - inet->inet_saddr = fl4->saddr; - } - -sk_rcv_saddr_set (sk, inet->inet_saddr); - -if (prev_addr_hashbucket) { - err =inet_bhash2_update_saddr (prev_addr_hashbucket, sk); + err =inet_bhash2_update_saddr (sk, &fl4->saddr,AF_INET );if (err) { - inet->inet_saddr =0 ; -sk_rcv_saddr_set (sk, prev_sk_rcv_saddr);ip_rt_put (rt);return err; } + }else { +sk_rcv_saddr_set (sk, inet->inet_saddr); } inet->inet_dport = usin->sin_port;net/dccp/ipv6.c +4
-19@@ -934,26 +934,11 @@ static int dccp_v6_connect(struct sock *sk, struct sockaddr *uaddr, }if (saddr =NULL ) { -struct inet_bind_hashbucket *prev_addr_hashbucket =NULL ; -struct in6_addr prev_v6_rcv_saddr; - -if (icsk->icsk_bind2_hash) { - prev_addr_hashbucket =inet_bhashfn_portaddr (&dccp_hashinfo, - sk,sock_net (sk), - inet->inet_num); - prev_v6_rcv_saddr = sk->sk_v6_rcv_saddr; - } - saddr = &fl6.saddr; - sk->sk_v6_rcv_saddr = *saddr; - -if (prev_addr_hashbucket) { - err =inet_bhash2_update_saddr (prev_addr_hashbucket, sk); -if (err) { - sk->sk_v6_rcv_saddr = prev_v6_rcv_saddr; - goto failure; - } - } + + err =inet_bhash2_update_saddr (sk, saddr,AF_INET6 ); +if (err) + goto failure; }/* set the source address */ net/ipv4/af_inet.c +1
-9@@ -1230,7 +1230,6 @@ EXPORT_SYMBOL(inet_unregister_protosw);static int inet_sk_reselect_saddr (struct sock *sk) { -struct inet_bind_hashbucket *prev_addr_hashbucket;struct inet_sock *inet =inet_sk (sk);__be32 old_saddr = inet->inet_saddr;__be32 daddr = inet->inet_daddr;@@ -1260,16 +1259,8 @@ static int inet_sk_reselect_saddr(struct sock *sk)return 0 ; } - prev_addr_hashbucket > -inet_bhashfn_portaddr (tcp_or_dccp_get_hashinfo (sk), sk, -sock_net (sk), inet->inet_num); - - inet->inet_saddr = inet->inet_rcv_saddr = new_saddr; - - err =inet_bhash2_update_saddr (prev_addr_hashbucket, sk); + err =inet_bhash2_update_saddr (sk, &new_saddr,AF_INET );if (err) { - inet->inet_saddr = old_saddr; - inet->inet_rcv_saddr = old_saddr;ip_rt_put (rt);return err; }net/ipv4/inet_hashtables.c +15
-1@@ -858,31 +858,65 @@ inet_bhash2_addr_any_hashbucket(const struct sock *sk, const struct net *net, inreturn &hinfo->bhash2[hash & (hinfo->bhash_size -1 )]; } -int inet_bhash2_update_saddr (struct inet_bind_hashbucket *prev_saddr,struct sock *sk) +static void inet_update_saddr (struct sock *sk,void *saddr,int family) +{ +if (family =AF_INET ) { +inet_sk (sk)->inet_saddr = *(__be32 *)saddr; +sk_rcv_saddr_set (sk,inet_sk (sk)->inet_saddr); + } +#if IS_ENABLED (CONFIG_IPV6 ) + else { + sk->sk_v6_rcv_saddr = *(struct in6_addr *)saddr; + } +#endif +} + +int inet_bhash2_update_saddr (struct sock *sk,void *saddr,int family) {struct inet_hashinfo *hinfo =tcp_or_dccp_get_hashinfo (sk); +struct inet_bind_hashbucket *head, *head2;struct inet_bind2_bucket *tb2, *new_tb2;int l3mdev =inet_sk_bound_l3mdev (sk);
net/ipv4/inet_hashtables.c inet_bhash2_addr_any_hashbucket(const struct sock *sk, const struct net *net, in · line 888
Quoted 10 lines
net/ipv4/inet_hashtables.c +7
-1-struct inet_bind_hashbucket *head2;int port =inet_sk (sk)->inet_num;struct net *net =sock_net (sk); +int bhash, err =0 ; + +if (!inet_csk (sk)->icsk_bind2_hash) { +/* Not bind()ed before. */ +inet_update_saddr (sk, saddr, family); + goto out; + }
I think it would be cleaner if this logic were outside bhash2_update_saddr(), since this mutates the sk's address when the socket hasn't been previously bound to bhash2. I think something like this would be clearer:
static int inet_update_saddr(struct sock *sk, void *saddr, int family)
{
if (!inet_csk(sk)->icsk_bind2_hash) {
update_sk_saddr(sk, saddr, family)
return 0;
}
return inet_bhash2_update_saddr(sk, saddr, family);
}and then from dccp/tcp_ipv4/6_connect(), we just call inet_update_saddr(). This also "moves" the lower-level implementation details (eg underlying bind tables) to inet_hashtables.c, instead of it being mentioned in the higher dccp_tcp_ipv4/6 layers.
What are your thoughts?
net/ipv4/inet_hashtables.c inet_bhash2_addr_any_hashbucket(const struct sock *sk, const struct net *net, in · line 896
Quoted 8 lines
net/ipv4/inet_hashtables.c +8
-0+ + bhash =inet_bhashfn (net, port, hinfo->bhash_size); + head = &hinfo->bhash[bhash]; + +/* If we change saddr locklessly, another thread +* iterating over bhash might see corrupted address. +*/ +spin_lock_bh (&head->lock);
I don't think we should be acquiring the bhash lock here. I think we only need to acquire it right before we mutate the saddr, and we can release it right after.
Quoted 133 lines
net/ipv4/inet_hashtables.c +16
-7/* Allocate a bind2 bucket ahead of time to avoid permanently putting * the bhash2 table in an inconsistent state if a new tb2 bucket * allocation fails. */ new_tb2 =kmem_cache_alloc (hinfo->bind2_bucket_cachep,GFP_ATOMIC ); -if (!new_tb2) -return -ENOMEM ; +if (!new_tb2) { + err = -ENOMEM ; + goto unlock; + } head2 =inet_bhashfn_portaddr (hinfo, sk, net, port); -spin_lock_bh (&prev_saddr->lock); +spin_lock (&head2->lock);__sk_del_bind2_node (sk);inet_bind2_bucket_destroy (hinfo->bind2_bucket_cachep,inet_csk (sk)->icsk_bind2_hash); -spin_unlock_bh (&prev_saddr->lock); +spin_unlock (&head2->lock); + +inet_update_saddr (sk, saddr, family); -spin_lock_bh (&head2->lock); + head2 =inet_bhashfn_portaddr (hinfo, sk, net, port); + +spin_lock (&head2->lock); tb2 =inet_bind2_bucket_find (head2, net, port, l3mdev, sk);if (!tb2) { tb2 = new_tb2;@@ -890,12 +924,15 @@ int inet_bhash2_update_saddr(struct inet_bind_hashbucket *prev_saddr, struct soc }sk_add_bind2_node (sk, &tb2->owners);inet_csk (sk)->icsk_bind2_hash = tb2; -spin_unlock_bh (&head2->lock); +spin_unlock (&head2->lock);if (tb2 != new_tb2)kmem_cache_free (hinfo->bind2_bucket_cachep, new_tb2); -return 0 ; +unlock: +spin_unlock_bh (&head->lock); +out: +return err; }EXPORT_SYMBOL_GPL (inet_bhash2_update_saddr);net/ipv4/tcp_ipv4.c +4
-16@@ -199,15 +199,14 @@ static int tcp_v4_pre_connect(struct sock *sk, struct sockaddr *uaddr,/* This will initiate an outgoing connection. */ int tcp_v4_connect (struct sock *sk,struct sockaddr *uaddr,int addr_len) { -struct inet_bind_hashbucket *prev_addr_hashbucket =NULL ;struct sockaddr_in *usin = (struct sockaddr_in *)uaddr;struct inet_timewait_death_row *tcp_death_row; -__be32 daddr, nexthop, prev_sk_rcv_saddr;struct inet_sock *inet =inet_sk (sk);struct tcp_sock *tp =tcp_sk (sk);struct ip_options_rcu *inet_opt;struct net *net =sock_net (sk);__be16 orig_sport, orig_dport; +__be32 daddr, nexthop;struct flowi4 *fl4;struct rtable *rt;int err;@@ -251,24 +250,13 @@ int tcp_v4_connect(struct sock *sk, struct sockaddr *uaddr, int addr_len) tcp_death_row = &sock_net (sk)->ipv4.tcp_death_row;if (!inet->inet_saddr) { -if (inet_csk (sk)->icsk_bind2_hash) { - prev_addr_hashbucket =inet_bhashfn_portaddr (tcp_death_row->hashinfo, - sk, net, inet->inet_num); - prev_sk_rcv_saddr = sk->sk_rcv_saddr; - } - inet->inet_saddr = fl4->saddr; - } - -sk_rcv_saddr_set (sk, inet->inet_saddr); - -if (prev_addr_hashbucket) { - err =inet_bhash2_update_saddr (prev_addr_hashbucket, sk); + err =inet_bhash2_update_saddr (sk, &fl4->saddr,AF_INET );if (err) { - inet->inet_saddr =0 ; -sk_rcv_saddr_set (sk, prev_sk_rcv_saddr);ip_rt_put (rt);return err; } + }else { +sk_rcv_saddr_set (sk, inet->inet_saddr); }if (tp->rx_opt.ts_recent_stamp && inet->inet_daddr != daddr) {net/ipv6/tcp_ipv6.c +3
-17@@ -292,24 +292,11 @@ static int tcp_v6_connect(struct sock *sk, struct sockaddr *uaddr, tcp_death_row = &sock_net (sk)->ipv4.tcp_death_row;if (!saddr) { -struct inet_bind_hashbucket *prev_addr_hashbucket =NULL ; -struct in6_addr prev_v6_rcv_saddr; - -if (icsk->icsk_bind2_hash) { - prev_addr_hashbucket =inet_bhashfn_portaddr (tcp_death_row->hashinfo, - sk, net, inet->inet_num); - prev_v6_rcv_saddr = sk->sk_v6_rcv_saddr; - } saddr = &fl6.saddr; - sk->sk_v6_rcv_saddr = *saddr; -if (prev_addr_hashbucket) { - err =inet_bhash2_update_saddr (prev_addr_hashbucket, sk); -if (err) { - sk->sk_v6_rcv_saddr = prev_v6_rcv_saddr; - goto failure; - } - } + err =inet_bhash2_update_saddr (sk, saddr,AF_INET6 ); +if (err) + goto failure; }/* set the source address */ --2.30.2
Re: [PATCH v3 net 3/4] dccp/tcp: Update saddr under bhash's lock.
From: Joanne Koong <[email protected]> Date: Fri, 18 Nov 2022 15:20:20 -0800
Quoted 201 lines · Kuniyuki Iwashima
Quoted 6 lines
When we call
connect()for a socket bound to a wildcard address, we update saddr locklessly. However, it could result in a data race; another thread iterating over bhash might see a corrupted address.Let's update saddr under the bhash bucket's lock.
Thanks for the quick turnaround!
Quoted 166 lines
Fixes 3df80d9320bc ("[DCCP]: Introduce DCCPv6")Fixes 7c657876b63c ("[DCCP]: Initial implementation")Fixes 1da177e4c3f4 ("Linux-2.6.12-rc2")Signed-off-by Kuniyuki Iwashima <[email protected]>--- include/net/inet_hashtables.h | 2 +- net/dccp/ipv4.c | 22 +++----------- net/dccp/ipv6.c | 23 +++------------ net/ipv4/af_inet.c | 11 +------ net/ipv4/inet_hashtables.c | 55 +++++++++++++++++++++++++++++------ net/ipv4/tcp_ipv4.c | 20 +++---------- net/ipv6/tcp_ipv6.c | 19 ++---------- 7 files changed, 63 insertions(+), 89 deletions(-)include/net/inet_hashtables.h +1
-1@@ -281,7 +281,7 @@ inet_bhash2_addr_any_hashbucket(const struct sock *sk, const struct net *net, in* sk_v6_rcv_saddr (ipv6) changes after it has been binded. The socket's * rcv_saddr field should already have been updated when this is called. */ -int inet_bhash2_update_saddr (struct inet_bind_hashbucket *prev_saddr,struct sock *sk); +int inet_bhash2_update_saddr (struct sock *sk,void *saddr,int family);void inet_bind_hash (struct sock *sk,struct inet_bind_bucket *tb,struct inet_bind2_bucket *tb2,unsigned short port);net/dccp/ipv4.c +4
-17@@ -45,11 +45,10 @@ static unsigned int dccp_v4_pernet_id __read_mostly;int dccp_v4_connect (struct sock *sk,struct sockaddr *uaddr,int addr_len) {const struct sockaddr_in *usin = (struct sockaddr_in *)uaddr; -struct inet_bind_hashbucket *prev_addr_hashbucket =NULL ; -__be32 daddr, nexthop, prev_sk_rcv_saddr;struct inet_sock *inet =inet_sk (sk);struct dccp_sock *dp =dccp_sk (sk);__be16 orig_sport, orig_dport; +__be32 daddr, nexthop;struct flowi4 *fl4;struct rtable *rt;int err;@@ -91,26 +90,13 @@ int dccp_v4_connect(struct sock *sk, struct sockaddr *uaddr, int addr_len) daddr = fl4->daddr;if (inet->inet_saddr =0 ) { -if (inet_csk (sk)->icsk_bind2_hash) { - prev_addr_hashbucket > > - inet_bhashfn_portaddr(&dccp_hashinfo, sk, -sock_net (sk), - inet->inet_num); - prev_sk_rcv_saddr = sk->sk_rcv_saddr; - } - inet->inet_saddr = fl4->saddr; - } - -sk_rcv_saddr_set (sk, inet->inet_saddr); - -if (prev_addr_hashbucket) { - err =inet_bhash2_update_saddr (prev_addr_hashbucket, sk); + err =inet_bhash2_update_saddr (sk, &fl4->saddr,AF_INET );if (err) { - inet->inet_saddr =0 ; -sk_rcv_saddr_set (sk, prev_sk_rcv_saddr);ip_rt_put (rt);return err; } + }else { +sk_rcv_saddr_set (sk, inet->inet_saddr); } inet->inet_dport = usin->sin_port;net/dccp/ipv6.c +4
-19@@ -934,26 +934,11 @@ static int dccp_v6_connect(struct sock *sk, struct sockaddr *uaddr, }if (saddr =NULL ) { -struct inet_bind_hashbucket *prev_addr_hashbucket =NULL ; -struct in6_addr prev_v6_rcv_saddr; - -if (icsk->icsk_bind2_hash) { - prev_addr_hashbucket =inet_bhashfn_portaddr (&dccp_hashinfo, - sk,sock_net (sk), - inet->inet_num); - prev_v6_rcv_saddr = sk->sk_v6_rcv_saddr; - } - saddr = &fl6.saddr; - sk->sk_v6_rcv_saddr = *saddr; - -if (prev_addr_hashbucket) { - err =inet_bhash2_update_saddr (prev_addr_hashbucket, sk); -if (err) { - sk->sk_v6_rcv_saddr = prev_v6_rcv_saddr; - goto failure; - } - } + + err =inet_bhash2_update_saddr (sk, saddr,AF_INET6 ); +if (err) + goto failure; }/* set the source address */ net/ipv4/af_inet.c +1
-9@@ -1230,7 +1230,6 @@ EXPORT_SYMBOL(inet_unregister_protosw);static int inet_sk_reselect_saddr (struct sock *sk) { -struct inet_bind_hashbucket *prev_addr_hashbucket;struct inet_sock *inet =inet_sk (sk);__be32 old_saddr = inet->inet_saddr;__be32 daddr = inet->inet_daddr;@@ -1260,16 +1259,8 @@ static int inet_sk_reselect_saddr(struct sock *sk)return 0 ; } - prev_addr_hashbucket > > - inet_bhashfn_portaddr(tcp_or_dccp_get_hashinfo(sk), sk, -sock_net (sk), inet->inet_num); - - inet->inet_saddr = inet->inet_rcv_saddr = new_saddr; - - err =inet_bhash2_update_saddr (prev_addr_hashbucket, sk); + err =inet_bhash2_update_saddr (sk, &new_saddr,AF_INET );if (err) { - inet->inet_saddr = old_saddr; - inet->inet_rcv_saddr = old_saddr;ip_rt_put (rt);return err; }net/ipv4/inet_hashtables.c +15
-1@@ -858,31 +858,65 @@ inet_bhash2_addr_any_hashbucket(const struct sock *sk, const struct net *net, inreturn &hinfo->bhash2[hash & (hinfo->bhash_size -1 )]; } -int inet_bhash2_update_saddr (struct inet_bind_hashbucket *prev_saddr,struct sock *sk) +static void inet_update_saddr (struct sock *sk,void *saddr,int family) +{ +if (family =AF_INET ) { +inet_sk (sk)->inet_saddr = *(__be32 *)saddr; +sk_rcv_saddr_set (sk,inet_sk (sk)->inet_saddr); + } +#if IS_ENABLED (CONFIG_IPV6 ) + else { + sk->sk_v6_rcv_saddr = *(struct in6_addr *)saddr; + } +#endif +} + +int inet_bhash2_update_saddr (struct sock *sk,void *saddr,int family) {struct inet_hashinfo *hinfo =tcp_or_dccp_get_hashinfo (sk); +struct inet_bind_hashbucket *head, *head2;struct inet_bind2_bucket *tb2, *new_tb2;int l3mdev =inet_sk_bound_l3mdev (sk);net/ipv4/inet_hashtables.c inet_bhash2_addr_any_hashbucket(const struct sock *sk, const struct net *net, in · line 888
Quoted 10 lines
net/ipv4/inet_hashtables.c +7
-1-struct inet_bind_hashbucket *head2;int port =inet_sk (sk)->inet_num;struct net *net =sock_net (sk); +int bhash, err =0 ; + +if (!inet_csk (sk)->icsk_bind2_hash) { +/* Not bind()ed before. */ +inet_update_saddr (sk, saddr, family); + goto out; + }I think it would be cleaner if this logic were outside
bhash2_update_saddr(), since this mutates the sk's address when the socket hasn't been previously bound to bhash2. I think something like this would be clearer:static int
inet_update_saddr(struct sock *sk, void *saddr, int family){ if (!inet_csk(sk)->icsk_bind2_hash) { update_sk_saddr(sk, saddr, family) return 0; } return inet_bhash2_update_saddr(sk, saddr, family); }
Quoted 6 lines
and then from dccp/tcp_ipv4/6_connect(), we just call
inet_update_saddr(). This also "moves" the lower-level implementation details (eg underlying bind tables) toinet_hashtables.c, instead of it being mentioned in the higher dccp_tcp_ipv4/6 layers.What are your thoughts?
Sounds good!
I'll change them like above.
Quoted 13 lines
net/ipv4/inet_hashtables.c inet_bhash2_addr_any_hashbucket(const struct sock *sk, const struct net *net, in · line 896
Quoted 9 lines
net/ipv4/inet_hashtables.c +8
-0+ + bhash =inet_bhashfn (net, port, hinfo->bhash_size); + head = &hinfo->bhash[bhash]; + +/* If we change saddr locklessly, another thread +* iterating over bhash might see corrupted address. +*/ +spin_lock_bh (&head->lock);I don't think we should be acquiring the bhash lock here. I think we only need to acquire it right before we mutate the saddr, and we can release it right after.
Exactly, will move it down before "__"inet_update_saddr() :)
Thank you!
Quoted 131 lines
Quoted 131 lines
net/ipv4/inet_hashtables.c +16
-7/* Allocate a bind2 bucket ahead of time to avoid permanently putting * the bhash2 table in an inconsistent state if a new tb2 bucket * allocation fails. */ new_tb2 =kmem_cache_alloc (hinfo->bind2_bucket_cachep,GFP_ATOMIC ); -if (!new_tb2) -return -ENOMEM ; +if (!new_tb2) { + err = -ENOMEM ; + goto unlock; + } head2 =inet_bhashfn_portaddr (hinfo, sk, net, port); -spin_lock_bh (&prev_saddr->lock); +spin_lock (&head2->lock);__sk_del_bind2_node (sk);inet_bind2_bucket_destroy (hinfo->bind2_bucket_cachep,inet_csk (sk)->icsk_bind2_hash); -spin_unlock_bh (&prev_saddr->lock); +spin_unlock (&head2->lock); + +inet_update_saddr (sk, saddr, family); -spin_lock_bh (&head2->lock); + head2 =inet_bhashfn_portaddr (hinfo, sk, net, port); + +spin_lock (&head2->lock); tb2 =inet_bind2_bucket_find (head2, net, port, l3mdev, sk);if (!tb2) { tb2 = new_tb2;@@ -890,12 +924,15 @@ int inet_bhash2_update_saddr(struct inet_bind_hashbucket *prev_saddr, struct soc }sk_add_bind2_node (sk, &tb2->owners);inet_csk (sk)->icsk_bind2_hash = tb2; -spin_unlock_bh (&head2->lock); +spin_unlock (&head2->lock);if (tb2 != new_tb2)kmem_cache_free (hinfo->bind2_bucket_cachep, new_tb2); -return 0 ; +unlock: +spin_unlock_bh (&head->lock); +out: +return err; }EXPORT_SYMBOL_GPL (inet_bhash2_update_saddr);net/ipv4/tcp_ipv4.c +4
-16@@ -199,15 +199,14 @@ static int tcp_v4_pre_connect(struct sock *sk, struct sockaddr *uaddr,/* This will initiate an outgoing connection. */ int tcp_v4_connect (struct sock *sk,struct sockaddr *uaddr,int addr_len) { -struct inet_bind_hashbucket *prev_addr_hashbucket =NULL ;struct sockaddr_in *usin = (struct sockaddr_in *)uaddr;struct inet_timewait_death_row *tcp_death_row; -__be32 daddr, nexthop, prev_sk_rcv_saddr;struct inet_sock *inet =inet_sk (sk);struct tcp_sock *tp =tcp_sk (sk);struct ip_options_rcu *inet_opt;struct net *net =sock_net (sk);__be16 orig_sport, orig_dport; +__be32 daddr, nexthop;struct flowi4 *fl4;struct rtable *rt;int err;@@ -251,24 +250,13 @@ int tcp_v4_connect(struct sock *sk, struct sockaddr *uaddr, int addr_len) tcp_death_row = &sock_net (sk)->ipv4.tcp_death_row;if (!inet->inet_saddr) { -if (inet_csk (sk)->icsk_bind2_hash) { - prev_addr_hashbucket =inet_bhashfn_portaddr (tcp_death_row->hashinfo, - sk, net, inet->inet_num); - prev_sk_rcv_saddr = sk->sk_rcv_saddr; - } - inet->inet_saddr = fl4->saddr; - } - -sk_rcv_saddr_set (sk, inet->inet_saddr); - -if (prev_addr_hashbucket) { - err =inet_bhash2_update_saddr (prev_addr_hashbucket, sk); + err =inet_bhash2_update_saddr (sk, &fl4->saddr,AF_INET );if (err) { - inet->inet_saddr =0 ; -sk_rcv_saddr_set (sk, prev_sk_rcv_saddr);ip_rt_put (rt);return err; } + }else { +sk_rcv_saddr_set (sk, inet->inet_saddr); }if (tp->rx_opt.ts_recent_stamp && inet->inet_daddr != daddr) {net/ipv6/tcp_ipv6.c +3
-17@@ -292,24 +292,11 @@ static int tcp_v6_connect(struct sock *sk, struct sockaddr *uaddr, tcp_death_row = &sock_net (sk)->ipv4.tcp_death_row;if (!saddr) { -struct inet_bind_hashbucket *prev_addr_hashbucket =NULL ; -struct in6_addr prev_v6_rcv_saddr; - -if (icsk->icsk_bind2_hash) { - prev_addr_hashbucket =inet_bhashfn_portaddr (tcp_death_row->hashinfo, - sk, net, inet->inet_num); - prev_v6_rcv_saddr = sk->sk_v6_rcv_saddr; - } saddr = &fl6.saddr; - sk->sk_v6_rcv_saddr = *saddr; -if (prev_addr_hashbucket) { - err =inet_bhash2_update_saddr (prev_addr_hashbucket, sk); -if (err) { - sk->sk_v6_rcv_saddr = prev_v6_rcv_saddr; - goto failure; - } - } + err =inet_bhash2_update_saddr (sk, saddr,AF_INET6 ); +if (err) + goto failure; }/* set the source address */ --2.30.2
Re: [PATCH v3 net 3/4] dccp/tcp: Update saddr under bhash's lock.
From: Kuniyuki Iwashima <[email protected]> Date: Fri, 18 Nov 2022 15:57:53 -0800
Quoted 228 lines
From: Joanne Koong <[email protected]> Date: Fri, 18 Nov 2022 15:20:20 -0800Quoted 201 lines · Kuniyuki Iwashima
Quoted 6 lines
When we call
connect()for a socket bound to a wildcard address, we update saddr locklessly. However, it could result in a data race; another thread iterating over bhash might see a corrupted address.Let's update saddr under the bhash bucket's lock.
Thanks for the quick turnaround!
Quoted 166 lines
Fixes 3df80d9320bc ("[DCCP]: Introduce DCCPv6")Fixes 7c657876b63c ("[DCCP]: Initial implementation")Fixes 1da177e4c3f4 ("Linux-2.6.12-rc2")Signed-off-by Kuniyuki Iwashima <[email protected]>--- include/net/inet_hashtables.h | 2 +- net/dccp/ipv4.c | 22 +++----------- net/dccp/ipv6.c | 23 +++------------ net/ipv4/af_inet.c | 11 +------ net/ipv4/inet_hashtables.c | 55 +++++++++++++++++++++++++++++------ net/ipv4/tcp_ipv4.c | 20 +++---------- net/ipv6/tcp_ipv6.c | 19 ++---------- 7 files changed, 63 insertions(+), 89 deletions(-)include/net/inet_hashtables.h +1
-1@@ -281,7 +281,7 @@ inet_bhash2_addr_any_hashbucket(const struct sock *sk, const struct net *net, in* sk_v6_rcv_saddr (ipv6) changes after it has been binded. The socket's * rcv_saddr field should already have been updated when this is called. */ -int inet_bhash2_update_saddr (struct inet_bind_hashbucket *prev_saddr,struct sock *sk); +int inet_bhash2_update_saddr (struct sock *sk,void *saddr,int family);void inet_bind_hash (struct sock *sk,struct inet_bind_bucket *tb,struct inet_bind2_bucket *tb2,unsigned short port);net/dccp/ipv4.c +4
-17@@ -45,11 +45,10 @@ static unsigned int dccp_v4_pernet_id __read_mostly;int dccp_v4_connect (struct sock *sk,struct sockaddr *uaddr,int addr_len) {const struct sockaddr_in *usin = (struct sockaddr_in *)uaddr; -struct inet_bind_hashbucket *prev_addr_hashbucket =NULL ; -__be32 daddr, nexthop, prev_sk_rcv_saddr;struct inet_sock *inet =inet_sk (sk);struct dccp_sock *dp =dccp_sk (sk);__be16 orig_sport, orig_dport; +__be32 daddr, nexthop;struct flowi4 *fl4;struct rtable *rt;int err;@@ -91,26 +90,13 @@ int dccp_v4_connect(struct sock *sk, struct sockaddr *uaddr, int addr_len) daddr = fl4->daddr;if (inet->inet_saddr =0 ) { -if (inet_csk (sk)->icsk_bind2_hash) { - prev_addr_hashbucket > > > - inet_bhashfn_portaddr(&dccp_hashinfo, sk, -sock_net (sk), - inet->inet_num); - prev_sk_rcv_saddr = sk->sk_rcv_saddr; - } - inet->inet_saddr = fl4->saddr; - } - -sk_rcv_saddr_set (sk, inet->inet_saddr); - -if (prev_addr_hashbucket) { - err =inet_bhash2_update_saddr (prev_addr_hashbucket, sk); + err =inet_bhash2_update_saddr (sk, &fl4->saddr,AF_INET );if (err) { - inet->inet_saddr =0 ; -sk_rcv_saddr_set (sk, prev_sk_rcv_saddr);ip_rt_put (rt);return err; } + }else { +sk_rcv_saddr_set (sk, inet->inet_saddr); } inet->inet_dport = usin->sin_port;net/dccp/ipv6.c +4
-19@@ -934,26 +934,11 @@ static int dccp_v6_connect(struct sock *sk, struct sockaddr *uaddr, }if (saddr =NULL ) { -struct inet_bind_hashbucket *prev_addr_hashbucket =NULL ; -struct in6_addr prev_v6_rcv_saddr; - -if (icsk->icsk_bind2_hash) { - prev_addr_hashbucket =inet_bhashfn_portaddr (&dccp_hashinfo, - sk,sock_net (sk), - inet->inet_num); - prev_v6_rcv_saddr = sk->sk_v6_rcv_saddr; - } - saddr = &fl6.saddr; - sk->sk_v6_rcv_saddr = *saddr; - -if (prev_addr_hashbucket) { - err =inet_bhash2_update_saddr (prev_addr_hashbucket, sk); -if (err) { - sk->sk_v6_rcv_saddr = prev_v6_rcv_saddr; - goto failure; - } - } + + err =inet_bhash2_update_saddr (sk, saddr,AF_INET6 ); +if (err) + goto failure; }/* set the source address */ net/ipv4/af_inet.c +1
-9@@ -1230,7 +1230,6 @@ EXPORT_SYMBOL(inet_unregister_protosw);static int inet_sk_reselect_saddr (struct sock *sk) { -struct inet_bind_hashbucket *prev_addr_hashbucket;struct inet_sock *inet =inet_sk (sk);__be32 old_saddr = inet->inet_saddr;__be32 daddr = inet->inet_daddr;@@ -1260,16 +1259,8 @@ static int inet_sk_reselect_saddr(struct sock *sk)return 0 ; } - prev_addr_hashbucket > > > - inet_bhashfn_portaddr(tcp_or_dccp_get_hashinfo(sk), sk, -sock_net (sk), inet->inet_num); - - inet->inet_saddr = inet->inet_rcv_saddr = new_saddr; - - err =inet_bhash2_update_saddr (prev_addr_hashbucket, sk); + err =inet_bhash2_update_saddr (sk, &new_saddr,AF_INET );if (err) { - inet->inet_saddr = old_saddr; - inet->inet_rcv_saddr = old_saddr;ip_rt_put (rt);return err; }net/ipv4/inet_hashtables.c +15
-1@@ -858,31 +858,65 @@ inet_bhash2_addr_any_hashbucket(const struct sock *sk, const struct net *net, inreturn &hinfo->bhash2[hash & (hinfo->bhash_size -1 )]; } -int inet_bhash2_update_saddr (struct inet_bind_hashbucket *prev_saddr,struct sock *sk) +static void inet_update_saddr (struct sock *sk,void *saddr,int family) +{ +if (family =AF_INET ) { +inet_sk (sk)->inet_saddr = *(__be32 *)saddr; +sk_rcv_saddr_set (sk,inet_sk (sk)->inet_saddr); + } +#if IS_ENABLED (CONFIG_IPV6 ) + else { + sk->sk_v6_rcv_saddr = *(struct in6_addr *)saddr; + } +#endif +} + +int inet_bhash2_update_saddr (struct sock *sk,void *saddr,int family) {struct inet_hashinfo *hinfo =tcp_or_dccp_get_hashinfo (sk); +struct inet_bind_hashbucket *head, *head2;struct inet_bind2_bucket *tb2, *new_tb2;int l3mdev =inet_sk_bound_l3mdev (sk);net/ipv4/inet_hashtables.c inet_bhash2_addr_any_hashbucket(const struct sock *sk, const struct net *net, in · line 888
Quoted 10 lines
net/ipv4/inet_hashtables.c +7
-1-struct inet_bind_hashbucket *head2;int port =inet_sk (sk)->inet_num;struct net *net =sock_net (sk); +int bhash, err =0 ; + +if (!inet_csk (sk)->icsk_bind2_hash) { +/* Not bind()ed before. */ +inet_update_saddr (sk, saddr, family); + goto out; + }I think it would be cleaner if this logic were outside
bhash2_update_saddr(), since this mutates the sk's address when the socket hasn't been previously bound to bhash2. I think something like this would be clearer:static int
inet_update_saddr(struct sock *sk, void *saddr, int family){ if (!inet_csk(sk)->icsk_bind2_hash) { update_sk_saddr(sk, saddr, family) return 0; } return inet_bhash2_update_saddr(sk, saddr, family); }Quoted 6 lines
and then from dccp/tcp_ipv4/6_connect(), we just call
inet_update_saddr(). This also "moves" the lower-level implementation details (eg underlying bind tables) toinet_hashtables.c, instead of it being mentioned in the higher dccp_tcp_ipv4/6 layers.What are your thoughts?
Sounds good!
I'll change them like above.Quoted 13 lines
net/ipv4/inet_hashtables.c inet_bhash2_addr_any_hashbucket(const struct sock *sk, const struct net *net, in · line 896
Quoted 9 lines
net/ipv4/inet_hashtables.c +8
-0+ + bhash =inet_bhashfn (net, port, hinfo->bhash_size); + head = &hinfo->bhash[bhash]; + +/* If we change saddr locklessly, another thread +* iterating over bhash might see corrupted address. +*/ +spin_lock_bh (&head->lock);I don't think we should be acquiring the bhash lock here. I think we only need to acquire it right before we mutate the saddr, and we can release it right after.
Quoted 1 line
Exactly, will move it down before "__"inet_update_saddr() :)
And I'll move this down after the first inet_bhashfn_portaddr() to align the same lock semantics with these 3 functions which aquire the bhash's lock before updating a bhash2 bucket.
* inet_csk_get_port()
* __inet_hash_connect()
* inet_put_port()
Quoted 135 lines
Thank you!
Quoted 131 lines
Quoted 131 lines
net/ipv4/inet_hashtables.c +16
-7/* Allocate a bind2 bucket ahead of time to avoid permanently putting * the bhash2 table in an inconsistent state if a new tb2 bucket * allocation fails. */ new_tb2 =kmem_cache_alloc (hinfo->bind2_bucket_cachep,GFP_ATOMIC ); -if (!new_tb2) -return -ENOMEM ; +if (!new_tb2) { + err = -ENOMEM ; + goto unlock; + } head2 =inet_bhashfn_portaddr (hinfo, sk, net, port); -spin_lock_bh (&prev_saddr->lock); +spin_lock (&head2->lock);__sk_del_bind2_node (sk);inet_bind2_bucket_destroy (hinfo->bind2_bucket_cachep,inet_csk (sk)->icsk_bind2_hash); -spin_unlock_bh (&prev_saddr->lock); +spin_unlock (&head2->lock); + +inet_update_saddr (sk, saddr, family); -spin_lock_bh (&head2->lock); + head2 =inet_bhashfn_portaddr (hinfo, sk, net, port); + +spin_lock (&head2->lock); tb2 =inet_bind2_bucket_find (head2, net, port, l3mdev, sk);if (!tb2) { tb2 = new_tb2;@@ -890,12 +924,15 @@ int inet_bhash2_update_saddr(struct inet_bind_hashbucket *prev_saddr, struct soc }sk_add_bind2_node (sk, &tb2->owners);inet_csk (sk)->icsk_bind2_hash = tb2; -spin_unlock_bh (&head2->lock); +spin_unlock (&head2->lock);if (tb2 != new_tb2)kmem_cache_free (hinfo->bind2_bucket_cachep, new_tb2); -return 0 ; +unlock: +spin_unlock_bh (&head->lock); +out: +return err; }EXPORT_SYMBOL_GPL (inet_bhash2_update_saddr);net/ipv4/tcp_ipv4.c +4
-16@@ -199,15 +199,14 @@ static int tcp_v4_pre_connect(struct sock *sk, struct sockaddr *uaddr,/* This will initiate an outgoing connection. */ int tcp_v4_connect (struct sock *sk,struct sockaddr *uaddr,int addr_len) { -struct inet_bind_hashbucket *prev_addr_hashbucket =NULL ;struct sockaddr_in *usin = (struct sockaddr_in *)uaddr;struct inet_timewait_death_row *tcp_death_row; -__be32 daddr, nexthop, prev_sk_rcv_saddr;struct inet_sock *inet =inet_sk (sk);struct tcp_sock *tp =tcp_sk (sk);struct ip_options_rcu *inet_opt;struct net *net =sock_net (sk);__be16 orig_sport, orig_dport; +__be32 daddr, nexthop;struct flowi4 *fl4;struct rtable *rt;int err;@@ -251,24 +250,13 @@ int tcp_v4_connect(struct sock *sk, struct sockaddr *uaddr, int addr_len) tcp_death_row = &sock_net (sk)->ipv4.tcp_death_row;if (!inet->inet_saddr) { -if (inet_csk (sk)->icsk_bind2_hash) { - prev_addr_hashbucket =inet_bhashfn_portaddr (tcp_death_row->hashinfo, - sk, net, inet->inet_num); - prev_sk_rcv_saddr = sk->sk_rcv_saddr; - } - inet->inet_saddr = fl4->saddr; - } - -sk_rcv_saddr_set (sk, inet->inet_saddr); - -if (prev_addr_hashbucket) { - err =inet_bhash2_update_saddr (prev_addr_hashbucket, sk); + err =inet_bhash2_update_saddr (sk, &fl4->saddr,AF_INET );if (err) { - inet->inet_saddr =0 ; -sk_rcv_saddr_set (sk, prev_sk_rcv_saddr);ip_rt_put (rt);return err; } + }else { +sk_rcv_saddr_set (sk, inet->inet_saddr); }if (tp->rx_opt.ts_recent_stamp && inet->inet_daddr != daddr) {net/ipv6/tcp_ipv6.c +3
-17@@ -292,24 +292,11 @@ static int tcp_v6_connect(struct sock *sk, struct sockaddr *uaddr, tcp_death_row = &sock_net (sk)->ipv4.tcp_death_row;if (!saddr) { -struct inet_bind_hashbucket *prev_addr_hashbucket =NULL ; -struct in6_addr prev_v6_rcv_saddr; - -if (icsk->icsk_bind2_hash) { - prev_addr_hashbucket =inet_bhashfn_portaddr (tcp_death_row->hashinfo, - sk, net, inet->inet_num); - prev_v6_rcv_saddr = sk->sk_v6_rcv_saddr; - } saddr = &fl6.saddr; - sk->sk_v6_rcv_saddr = *saddr; -if (prev_addr_hashbucket) { - err =inet_bhash2_update_saddr (prev_addr_hashbucket, sk); -if (err) { - sk->sk_v6_rcv_saddr = prev_v6_rcv_saddr; - goto failure; - } - } + err =inet_bhash2_update_saddr (sk, saddr,AF_INET6 ); +if (err) + goto failure; }/* set the source address */ --2.30.2
Re: [PATCH v3 net 3/4] dccp/tcp: Update saddr under bhash's lock.
In reply to Joanne KoongFrom: Joanne Koong <[email protected]> Date: Fri, 18 Nov 2022 15:20:20 -0800
Quoted 201 lines · Kuniyuki Iwashima
Quoted 6 lines
When we call
connect()for a socket bound to a wildcard address, we update saddr locklessly. However, it could result in a data race; another thread iterating over bhash might see a corrupted address.Let's update saddr under the bhash bucket's lock.
Thanks for the quick turnaround!
Quoted 166 lines
Fixes 3df80d9320bc ("[DCCP]: Introduce DCCPv6")Fixes 7c657876b63c ("[DCCP]: Initial implementation")Fixes 1da177e4c3f4 ("Linux-2.6.12-rc2")Signed-off-by Kuniyuki Iwashima <[email protected]>--- include/net/inet_hashtables.h | 2 +- net/dccp/ipv4.c | 22 +++----------- net/dccp/ipv6.c | 23 +++------------ net/ipv4/af_inet.c | 11 +------ net/ipv4/inet_hashtables.c | 55 +++++++++++++++++++++++++++++------ net/ipv4/tcp_ipv4.c | 20 +++---------- net/ipv6/tcp_ipv6.c | 19 ++---------- 7 files changed, 63 insertions(+), 89 deletions(-)include/net/inet_hashtables.h +1
-1@@ -281,7 +281,7 @@ inet_bhash2_addr_any_hashbucket(const struct sock *sk, const struct net *net, in* sk_v6_rcv_saddr (ipv6) changes after it has been binded. The socket's * rcv_saddr field should already have been updated when this is called. */ -int inet_bhash2_update_saddr (struct inet_bind_hashbucket *prev_saddr,struct sock *sk); +int inet_bhash2_update_saddr (struct sock *sk,void *saddr,int family);void inet_bind_hash (struct sock *sk,struct inet_bind_bucket *tb,struct inet_bind2_bucket *tb2,unsigned short port);net/dccp/ipv4.c +4
-17@@ -45,11 +45,10 @@ static unsigned int dccp_v4_pernet_id __read_mostly;int dccp_v4_connect (struct sock *sk,struct sockaddr *uaddr,int addr_len) {const struct sockaddr_in *usin = (struct sockaddr_in *)uaddr; -struct inet_bind_hashbucket *prev_addr_hashbucket =NULL ; -__be32 daddr, nexthop, prev_sk_rcv_saddr;struct inet_sock *inet =inet_sk (sk);struct dccp_sock *dp =dccp_sk (sk);__be16 orig_sport, orig_dport; +__be32 daddr, nexthop;struct flowi4 *fl4;struct rtable *rt;int err;@@ -91,26 +90,13 @@ int dccp_v4_connect(struct sock *sk, struct sockaddr *uaddr, int addr_len) daddr = fl4->daddr;if (inet->inet_saddr =0 ) { -if (inet_csk (sk)->icsk_bind2_hash) { - prev_addr_hashbucket > > - inet_bhashfn_portaddr(&dccp_hashinfo, sk, -sock_net (sk), - inet->inet_num); - prev_sk_rcv_saddr = sk->sk_rcv_saddr; - } - inet->inet_saddr = fl4->saddr; - } - -sk_rcv_saddr_set (sk, inet->inet_saddr); - -if (prev_addr_hashbucket) { - err =inet_bhash2_update_saddr (prev_addr_hashbucket, sk); + err =inet_bhash2_update_saddr (sk, &fl4->saddr,AF_INET );if (err) { - inet->inet_saddr =0 ; -sk_rcv_saddr_set (sk, prev_sk_rcv_saddr);ip_rt_put (rt);return err; } + }else { +sk_rcv_saddr_set (sk, inet->inet_saddr); } inet->inet_dport = usin->sin_port;net/dccp/ipv6.c +4
-19@@ -934,26 +934,11 @@ static int dccp_v6_connect(struct sock *sk, struct sockaddr *uaddr, }if (saddr =NULL ) { -struct inet_bind_hashbucket *prev_addr_hashbucket =NULL ; -struct in6_addr prev_v6_rcv_saddr; - -if (icsk->icsk_bind2_hash) { - prev_addr_hashbucket =inet_bhashfn_portaddr (&dccp_hashinfo, - sk,sock_net (sk), - inet->inet_num); - prev_v6_rcv_saddr = sk->sk_v6_rcv_saddr; - } - saddr = &fl6.saddr; - sk->sk_v6_rcv_saddr = *saddr; - -if (prev_addr_hashbucket) { - err =inet_bhash2_update_saddr (prev_addr_hashbucket, sk); -if (err) { - sk->sk_v6_rcv_saddr = prev_v6_rcv_saddr; - goto failure; - } - } + + err =inet_bhash2_update_saddr (sk, saddr,AF_INET6 ); +if (err) + goto failure; }/* set the source address */ net/ipv4/af_inet.c +1
-9@@ -1230,7 +1230,6 @@ EXPORT_SYMBOL(inet_unregister_protosw);static int inet_sk_reselect_saddr (struct sock *sk) { -struct inet_bind_hashbucket *prev_addr_hashbucket;struct inet_sock *inet =inet_sk (sk);__be32 old_saddr = inet->inet_saddr;__be32 daddr = inet->inet_daddr;@@ -1260,16 +1259,8 @@ static int inet_sk_reselect_saddr(struct sock *sk)return 0 ; } - prev_addr_hashbucket > > - inet_bhashfn_portaddr(tcp_or_dccp_get_hashinfo(sk), sk, -sock_net (sk), inet->inet_num); - - inet->inet_saddr = inet->inet_rcv_saddr = new_saddr; - - err =inet_bhash2_update_saddr (prev_addr_hashbucket, sk); + err =inet_bhash2_update_saddr (sk, &new_saddr,AF_INET );if (err) { - inet->inet_saddr = old_saddr; - inet->inet_rcv_saddr = old_saddr;ip_rt_put (rt);return err; }net/ipv4/inet_hashtables.c +15
-1@@ -858,31 +858,65 @@ inet_bhash2_addr_any_hashbucket(const struct sock *sk, const struct net *net, inreturn &hinfo->bhash2[hash & (hinfo->bhash_size -1 )]; } -int inet_bhash2_update_saddr (struct inet_bind_hashbucket *prev_saddr,struct sock *sk) +static void inet_update_saddr (struct sock *sk,void *saddr,int family) +{ +if (family =AF_INET ) { +inet_sk (sk)->inet_saddr = *(__be32 *)saddr; +sk_rcv_saddr_set (sk,inet_sk (sk)->inet_saddr); + } +#if IS_ENABLED (CONFIG_IPV6 ) + else { + sk->sk_v6_rcv_saddr = *(struct in6_addr *)saddr; + } +#endif +} + +int inet_bhash2_update_saddr (struct sock *sk,void *saddr,int family) {struct inet_hashinfo *hinfo =tcp_or_dccp_get_hashinfo (sk); +struct inet_bind_hashbucket *head, *head2;struct inet_bind2_bucket *tb2, *new_tb2;int l3mdev =inet_sk_bound_l3mdev (sk);net/ipv4/inet_hashtables.c inet_bhash2_addr_any_hashbucket(const struct sock *sk, const struct net *net, in · line 888
Quoted 10 lines
net/ipv4/inet_hashtables.c +7
-1-struct inet_bind_hashbucket *head2;int port =inet_sk (sk)->inet_num;struct net *net =sock_net (sk); +int bhash, err =0 ; + +if (!inet_csk (sk)->icsk_bind2_hash) { +/* Not bind()ed before. */ +inet_update_saddr (sk, saddr, family); + goto out; + }I think it would be cleaner if this logic were outside
bhash2_update_saddr(), since this mutates the sk's address when the socket hasn't been previously bound to bhash2. I think something like this would be clearer:static int
inet_update_saddr(struct sock *sk, void *saddr, int family){ if (!inet_csk(sk)->icsk_bind2_hash) { update_sk_saddr(sk, saddr, family) return 0; } return inet_bhash2_update_saddr(sk, saddr, family); }
Quoted 6 lines
and then from dccp/tcp_ipv4/6_connect(), we just call
inet_update_saddr(). This also "moves" the lower-level implementation details (eg underlying bind tables) toinet_hashtables.c, instead of it being mentioned in the higher dccp_tcp_ipv4/6 layers.What are your thoughts?
Hmm.. There are other users of inet_reset_saddr(). If we hide the details of reset & bhash2 logic too, we need a cleanup patch below.
So, I'll keep this part as is in the next spin. If needed, I can post a followup for net-next.
And I noticed UDP has the same problem, this will be another series.
Thank you!
---8<---
include/net/inet_hashtables.h +1 -0
@@ -277,6 +277,7 @@ inet_bhashfn_portaddr(const struct inet_hashinfo *hinfo, const struct sock *sk,struct inet_bind_hashbucket *inet_bhash2_addr_any_hashbucket (const struct sock *sk,const struct net *net,int port); +void inet_reset_saddr (struct sock *sk);int inet_update_saddr (struct sock *sk,void *saddr,int family); void inet_bind_hash(struct sock *sk, struct inet_bind_bucket *tb,
include/net/ip.h +1 -1
@@ -632,7 +632,7 @@ static inline void ip_ipgre_mc_map(__be32 naddr, const unsigned char *broadcast,#include <linux/ipv6.h> #endif -static __inline__void inet_reset_saddr (struct sock *sk) +static __inline__void __inet_reset_saddr (struct sock *sk) {inet_sk (sk)->inet_rcv_saddr =inet_sk (sk)->inet_saddr =0 ;#if IS_ENABLED (CONFIG_IPV6 )
net/ipv4/inet_hashtables.c +6 -0
@@ -941,6 +941,12 @@ int inet_update_saddr(struct sock *sk, void *saddr, int family) }EXPORT_SYMBOL_GPL (inet_update_saddr); +void inet_reset_saddr (struct sock *sk) +{ +__inet_reset_saddr (sk); +} +EXPORT_SYMBOL_GPL (inet_reset_saddr); +/* RFC 6056 3.3.4. Algorithm 4: Double-Hash Port Selection Algorithm * Note that we use 32bit integers (vs RFC 'short integers') * because 2^16 is not a multiple of num_ephemeral and this
net/ipv4/udp.c +1 -1
@@ -1971,7 +1971,7 @@ int __udp_disconnect(struct sock *sk, int flags)sock_rps_reset_rxhash (sk); sk->sk_bound_dev_if =0 ;if (!(sk->sk_userlocks &SOCK_BINDADDR_LOCK )) { -inet_reset_saddr (sk); +__inet_reset_saddr (sk);if (sk->sk_prot->rehash && (sk->sk_userlocks &SOCK_BINDPORT_LOCK )) sk->sk_prot->rehash (sk);
net/ipv6/af_inet6.c +2 -2
@@ -411,7 +411,7 @@ static int __inet6_bind(struct sock *sk, struct sockaddr *uaddr, int addr_len, (flags & BIND_FORCE_ADDRESS_NO_PORT))) {if (sk->sk_prot->get_port (sk, snum)) { sk->sk_ipv6only = saved_ipv6only; -inet_reset_saddr (sk); +__inet_reset_saddr (sk); err = -EADDRINUSE ; goto out; }
@@ -419,7 +419,7 @@ static int __inet6_bind(struct sock *sk, struct sockaddr *uaddr, int addr_len, err =BPF_CGROUP_RUN_PROG_INET6_POST_BIND (sk);if (err) { sk->sk_ipv6only = saved_ipv6only; -inet_reset_saddr (sk); +__inet_reset_saddr (sk);if (sk->sk_prot->put_port) sk->sk_prot->put_port (sk); goto out;
---8<---
Quoted 146 lines
net/ipv4/inet_hashtables.c inet_bhash2_addr_any_hashbucket(const struct sock *sk, const struct net *net, in · line 896
Quoted 9 lines
net/ipv4/inet_hashtables.c +8
-0+ + bhash =inet_bhashfn (net, port, hinfo->bhash_size); + head = &hinfo->bhash[bhash]; + +/* If we change saddr locklessly, another thread +* iterating over bhash might see corrupted address. +*/ +spin_lock_bh (&head->lock);I don't think we should be acquiring the bhash lock here. I think we only need to acquire it right before we mutate the saddr, and we can release it right after.
Quoted 132 lines
net/ipv4/inet_hashtables.c +16
-7/* Allocate a bind2 bucket ahead of time to avoid permanently putting * the bhash2 table in an inconsistent state if a new tb2 bucket * allocation fails. */ new_tb2 =kmem_cache_alloc (hinfo->bind2_bucket_cachep,GFP_ATOMIC ); -if (!new_tb2) -return -ENOMEM ; +if (!new_tb2) { + err = -ENOMEM ; + goto unlock; + } head2 =inet_bhashfn_portaddr (hinfo, sk, net, port); -spin_lock_bh (&prev_saddr->lock); +spin_lock (&head2->lock);__sk_del_bind2_node (sk);inet_bind2_bucket_destroy (hinfo->bind2_bucket_cachep,inet_csk (sk)->icsk_bind2_hash); -spin_unlock_bh (&prev_saddr->lock); +spin_unlock (&head2->lock); + +inet_update_saddr (sk, saddr, family); -spin_lock_bh (&head2->lock); + head2 =inet_bhashfn_portaddr (hinfo, sk, net, port); + +spin_lock (&head2->lock); tb2 =inet_bind2_bucket_find (head2, net, port, l3mdev, sk);if (!tb2) { tb2 = new_tb2;@@ -890,12 +924,15 @@ int inet_bhash2_update_saddr(struct inet_bind_hashbucket *prev_saddr, struct soc }sk_add_bind2_node (sk, &tb2->owners);inet_csk (sk)->icsk_bind2_hash = tb2; -spin_unlock_bh (&head2->lock); +spin_unlock (&head2->lock);if (tb2 != new_tb2)kmem_cache_free (hinfo->bind2_bucket_cachep, new_tb2); -return 0 ; +unlock: +spin_unlock_bh (&head->lock); +out: +return err; }EXPORT_SYMBOL_GPL (inet_bhash2_update_saddr);net/ipv4/tcp_ipv4.c +4
-16@@ -199,15 +199,14 @@ static int tcp_v4_pre_connect(struct sock *sk, struct sockaddr *uaddr,/* This will initiate an outgoing connection. */ int tcp_v4_connect (struct sock *sk,struct sockaddr *uaddr,int addr_len) { -struct inet_bind_hashbucket *prev_addr_hashbucket =NULL ;struct sockaddr_in *usin = (struct sockaddr_in *)uaddr;struct inet_timewait_death_row *tcp_death_row; -__be32 daddr, nexthop, prev_sk_rcv_saddr;struct inet_sock *inet =inet_sk (sk);struct tcp_sock *tp =tcp_sk (sk);struct ip_options_rcu *inet_opt;struct net *net =sock_net (sk);__be16 orig_sport, orig_dport; +__be32 daddr, nexthop;struct flowi4 *fl4;struct rtable *rt;int err;@@ -251,24 +250,13 @@ int tcp_v4_connect(struct sock *sk, struct sockaddr *uaddr, int addr_len) tcp_death_row = &sock_net (sk)->ipv4.tcp_death_row;if (!inet->inet_saddr) { -if (inet_csk (sk)->icsk_bind2_hash) { - prev_addr_hashbucket =inet_bhashfn_portaddr (tcp_death_row->hashinfo, - sk, net, inet->inet_num); - prev_sk_rcv_saddr = sk->sk_rcv_saddr; - } - inet->inet_saddr = fl4->saddr; - } - -sk_rcv_saddr_set (sk, inet->inet_saddr); - -if (prev_addr_hashbucket) { - err =inet_bhash2_update_saddr (prev_addr_hashbucket, sk); + err =inet_bhash2_update_saddr (sk, &fl4->saddr,AF_INET );if (err) { - inet->inet_saddr =0 ; -sk_rcv_saddr_set (sk, prev_sk_rcv_saddr);ip_rt_put (rt);return err; } + }else { +sk_rcv_saddr_set (sk, inet->inet_saddr); }if (tp->rx_opt.ts_recent_stamp && inet->inet_daddr != daddr) {net/ipv6/tcp_ipv6.c +3
-17@@ -292,24 +292,11 @@ static int tcp_v6_connect(struct sock *sk, struct sockaddr *uaddr, tcp_death_row = &sock_net (sk)->ipv4.tcp_death_row;if (!saddr) { -struct inet_bind_hashbucket *prev_addr_hashbucket =NULL ; -struct in6_addr prev_v6_rcv_saddr; - -if (icsk->icsk_bind2_hash) { - prev_addr_hashbucket =inet_bhashfn_portaddr (tcp_death_row->hashinfo, - sk, net, inet->inet_num); - prev_v6_rcv_saddr = sk->sk_v6_rcv_saddr; - } saddr = &fl6.saddr; - sk->sk_v6_rcv_saddr = *saddr; -if (prev_addr_hashbucket) { - err =inet_bhash2_update_saddr (prev_addr_hashbucket, sk); -if (err) { - sk->sk_v6_rcv_saddr = prev_v6_rcv_saddr; - goto failure; - } - } + err =inet_bhash2_update_saddr (sk, saddr,AF_INET6 ); +if (err) + goto failure; }/* set the source address */ --2.30.2
Re: [PATCH v3 net 3/4] dccp/tcp: Update saddr under bhash's lock.
Quoted 209 lines · Kuniyuki Iwashima
From: Joanne Koong <[email protected]> Date: Fri, 18 Nov 2022 15:20:20 -0800Quoted 206 lines · Kuniyuki Iwashima
Quoted 6 lines
When we call
connect()for a socket bound to a wildcard address, we update saddr locklessly. However, it could result in a data race; another thread iterating over bhash might see a corrupted address.Let's update saddr under the bhash bucket's lock.
Thanks for the quick turnaround!
Quoted 166 lines
Fixes 3df80d9320bc ("[DCCP]: Introduce DCCPv6")Fixes 7c657876b63c ("[DCCP]: Initial implementation")Fixes 1da177e4c3f4 ("Linux-2.6.12-rc2")Signed-off-by Kuniyuki Iwashima <[email protected]>--- include/net/inet_hashtables.h | 2 +- net/dccp/ipv4.c | 22 +++----------- net/dccp/ipv6.c | 23 +++------------ net/ipv4/af_inet.c | 11 +------ net/ipv4/inet_hashtables.c | 55 +++++++++++++++++++++++++++++------ net/ipv4/tcp_ipv4.c | 20 +++---------- net/ipv6/tcp_ipv6.c | 19 ++---------- 7 files changed, 63 insertions(+), 89 deletions(-)include/net/inet_hashtables.h +1
-1@@ -281,7 +281,7 @@ inet_bhash2_addr_any_hashbucket(const struct sock *sk, const struct net *net, in* sk_v6_rcv_saddr (ipv6) changes after it has been binded. The socket's * rcv_saddr field should already have been updated when this is called. */ -int inet_bhash2_update_saddr (struct inet_bind_hashbucket *prev_saddr,struct sock *sk); +int inet_bhash2_update_saddr (struct sock *sk,void *saddr,int family);void inet_bind_hash (struct sock *sk,struct inet_bind_bucket *tb,struct inet_bind2_bucket *tb2,unsigned short port);net/dccp/ipv4.c +4
-17@@ -45,11 +45,10 @@ static unsigned int dccp_v4_pernet_id __read_mostly;int dccp_v4_connect (struct sock *sk,struct sockaddr *uaddr,int addr_len) {const struct sockaddr_in *usin = (struct sockaddr_in *)uaddr; -struct inet_bind_hashbucket *prev_addr_hashbucket =NULL ; -__be32 daddr, nexthop, prev_sk_rcv_saddr;struct inet_sock *inet =inet_sk (sk);struct dccp_sock *dp =dccp_sk (sk);__be16 orig_sport, orig_dport; +__be32 daddr, nexthop;struct flowi4 *fl4;struct rtable *rt;int err;@@ -91,26 +90,13 @@ int dccp_v4_connect(struct sock *sk, struct sockaddr *uaddr, int addr_len) daddr = fl4->daddr;if (inet->inet_saddr =0 ) { -if (inet_csk (sk)->icsk_bind2_hash) { - prev_addr_hashbucket > > > - inet_bhashfn_portaddr(&dccp_hashinfo, sk, -sock_net (sk), - inet->inet_num); - prev_sk_rcv_saddr = sk->sk_rcv_saddr; - } - inet->inet_saddr = fl4->saddr; - } - -sk_rcv_saddr_set (sk, inet->inet_saddr); - -if (prev_addr_hashbucket) { - err =inet_bhash2_update_saddr (prev_addr_hashbucket, sk); + err =inet_bhash2_update_saddr (sk, &fl4->saddr,AF_INET );if (err) { - inet->inet_saddr =0 ; -sk_rcv_saddr_set (sk, prev_sk_rcv_saddr);ip_rt_put (rt);return err; } + }else { +sk_rcv_saddr_set (sk, inet->inet_saddr); } inet->inet_dport = usin->sin_port;net/dccp/ipv6.c +4
-19@@ -934,26 +934,11 @@ static int dccp_v6_connect(struct sock *sk, struct sockaddr *uaddr, }if (saddr =NULL ) { -struct inet_bind_hashbucket *prev_addr_hashbucket =NULL ; -struct in6_addr prev_v6_rcv_saddr; - -if (icsk->icsk_bind2_hash) { - prev_addr_hashbucket =inet_bhashfn_portaddr (&dccp_hashinfo, - sk,sock_net (sk), - inet->inet_num); - prev_v6_rcv_saddr = sk->sk_v6_rcv_saddr; - } - saddr = &fl6.saddr; - sk->sk_v6_rcv_saddr = *saddr; - -if (prev_addr_hashbucket) { - err =inet_bhash2_update_saddr (prev_addr_hashbucket, sk); -if (err) { - sk->sk_v6_rcv_saddr = prev_v6_rcv_saddr; - goto failure; - } - } + + err =inet_bhash2_update_saddr (sk, saddr,AF_INET6 ); +if (err) + goto failure; }/* set the source address */ net/ipv4/af_inet.c +1
-9@@ -1230,7 +1230,6 @@ EXPORT_SYMBOL(inet_unregister_protosw);static int inet_sk_reselect_saddr (struct sock *sk) { -struct inet_bind_hashbucket *prev_addr_hashbucket;struct inet_sock *inet =inet_sk (sk);__be32 old_saddr = inet->inet_saddr;__be32 daddr = inet->inet_daddr;@@ -1260,16 +1259,8 @@ static int inet_sk_reselect_saddr(struct sock *sk)return 0 ; } - prev_addr_hashbucket > > > - inet_bhashfn_portaddr(tcp_or_dccp_get_hashinfo(sk), sk, -sock_net (sk), inet->inet_num); - - inet->inet_saddr = inet->inet_rcv_saddr = new_saddr; - - err =inet_bhash2_update_saddr (prev_addr_hashbucket, sk); + err =inet_bhash2_update_saddr (sk, &new_saddr,AF_INET );if (err) { - inet->inet_saddr = old_saddr; - inet->inet_rcv_saddr = old_saddr;ip_rt_put (rt);return err; }net/ipv4/inet_hashtables.c +15
-1@@ -858,31 +858,65 @@ inet_bhash2_addr_any_hashbucket(const struct sock *sk, const struct net *net, inreturn &hinfo->bhash2[hash & (hinfo->bhash_size -1 )]; } -int inet_bhash2_update_saddr (struct inet_bind_hashbucket *prev_saddr,struct sock *sk) +static void inet_update_saddr (struct sock *sk,void *saddr,int family) +{ +if (family =AF_INET ) { +inet_sk (sk)->inet_saddr = *(__be32 *)saddr; +sk_rcv_saddr_set (sk,inet_sk (sk)->inet_saddr); + } +#if IS_ENABLED (CONFIG_IPV6 ) + else { + sk->sk_v6_rcv_saddr = *(struct in6_addr *)saddr; + } +#endif +} + +int inet_bhash2_update_saddr (struct sock *sk,void *saddr,int family) {struct inet_hashinfo *hinfo =tcp_or_dccp_get_hashinfo (sk); +struct inet_bind_hashbucket *head, *head2;struct inet_bind2_bucket *tb2, *new_tb2;int l3mdev =inet_sk_bound_l3mdev (sk);net/ipv4/inet_hashtables.c inet_bhash2_addr_any_hashbucket(const struct sock *sk, const struct net *net, in · line 888
Quoted 10 lines
net/ipv4/inet_hashtables.c +7
-1-struct inet_bind_hashbucket *head2;int port =inet_sk (sk)->inet_num;struct net *net =sock_net (sk); +int bhash, err =0 ; + +if (!inet_csk (sk)->icsk_bind2_hash) { +/* Not bind()ed before. */ +inet_update_saddr (sk, saddr, family); + goto out; + }I think it would be cleaner if this logic were outside
bhash2_update_saddr(), since this mutates the sk's address when the socket hasn't been previously bound to bhash2. I think something like this would be clearer:static int
inet_update_saddr(struct sock *sk, void *saddr, int family){ if (!inet_csk(sk)->icsk_bind2_hash) { update_sk_saddr(sk, saddr, family) return 0; } return inet_bhash2_update_saddr(sk, saddr, family); }and then from dccp/tcp_ipv4/6_connect(), we just call
inet_update_saddr(). This also "moves" the lower-level implementation details (eg underlying bind tables) toinet_hashtables.c, instead of it being mentioned in the higher dccp_tcp_ipv4/6 layers.
Quoted 4 lines
Quoted 1 line
What are your thoughts?
Hmm.. There are other users of
inet_reset_saddr(). If we hide the details of reset & bhash2 logic too, we need a cleanup patch below.
I don't think reset needs to be hidden / abstracted away, that sounds generic. But bhash2 sounds like an implementation detail, which is why it seems cleaner to me if we didn't have bhash2 mentioned in the dccp/tcp_ipv4/6 layers.
Quoted 5 lines
So, I'll keep this part as is in the next spin. If needed, I can post a followup for net-next.
And I noticed UDP has the same problem, this will be another series.
Sounds good, looking forward to your UDP patchset.
Quoted 231 lines
Thank you!
---8<---
include/net/inet_hashtables.h +1
-0@@ -277,6 +277,7 @@ inet_bhashfn_portaddr(const struct inet_hashinfo *hinfo, const struct sock *sk,struct inet_bind_hashbucket *inet_bhash2_addr_any_hashbucket (const struct sock *sk,const struct net *net,int port); +void inet_reset_saddr (struct sock *sk);int inet_update_saddr (struct sock *sk,void *saddr,int family); void inet_bind_hash(struct sock *sk, struct inet_bind_bucket *tb,include/net/ip.h +1
-1@@ -632,7 +632,7 @@ static inline void ip_ipgre_mc_map(__be32 naddr, const unsigned char *broadcast,#include <linux/ipv6.h> #endif -static __inline__void inet_reset_saddr (struct sock *sk) +static __inline__void __inet_reset_saddr (struct sock *sk) {inet_sk (sk)->inet_rcv_saddr =inet_sk (sk)->inet_saddr =0 ;#if IS_ENABLED (CONFIG_IPV6 )net/ipv4/inet_hashtables.c +6
-0@@ -941,6 +941,12 @@ int inet_update_saddr(struct sock *sk, void *saddr, int family) }EXPORT_SYMBOL_GPL (inet_update_saddr); +void inet_reset_saddr (struct sock *sk) +{ +__inet_reset_saddr (sk); +} +EXPORT_SYMBOL_GPL (inet_reset_saddr); +/* RFC 6056 3.3.4. Algorithm 4: Double-Hash Port Selection Algorithm * Note that we use 32bit integers (vs RFC 'short integers') * because 2^16 is not a multiple of num_ephemeral and this net/ipv4/udp.c +1
-1@@ -1971,7 +1971,7 @@ int __udp_disconnect(struct sock *sk, int flags)sock_rps_reset_rxhash (sk); sk->sk_bound_dev_if =0 ;if (!(sk->sk_userlocks &SOCK_BINDADDR_LOCK )) { -inet_reset_saddr (sk); +__inet_reset_saddr (sk);if (sk->sk_prot->rehash && (sk->sk_userlocks &SOCK_BINDPORT_LOCK )) sk->sk_prot->rehash (sk);net/ipv6/af_inet6.c +2
-3@@ -411,7 +411,7 @@ static int __inet6_bind(struct sock *sk, struct sockaddr *uaddr, int addr_len, (flags & BIND_FORCE_ADDRESS_NO_PORT))) {if (sk->sk_prot->get_port (sk, snum)) { sk->sk_ipv6only = saved_ipv6only; -inet_reset_saddr (sk); +__inet_reset_saddr (sk); err = -EADDRINUSE ; goto out; }@@ -419,7 +419,7 @@ static int __inet6_bind(struct sock *sk, struct sockaddr *uaddr, int addr_len, err =BPF_CGROUP_RUN_PROG_INET6_POST_BIND (sk);if (err) { sk->sk_ipv6only = saved_ipv6only; -inet_reset_saddr (sk); +__inet_reset_saddr (sk);if (sk->sk_prot->put_port) sk->sk_prot->put_port (sk); goto out; ---8<---Quoted 148 lines
net/ipv4/inet_hashtables.c inet_bhash2_addr_any_hashbucket(const struct sock *sk, const struct net *net, in · line 896
Quoted 11 lines
net/ipv4/inet_hashtables.c +8
-0+ + bhash =inet_bhashfn (net, port, hinfo->bhash_size); + head = &hinfo->bhash[bhash]; + +/* If we change saddr locklessly, another thread +* iterating over bhash might see corrupted address. +*/ +spin_lock_bh (&head->lock);I don't think we should be acquiring the bhash lock here. I think we only need to acquire it right before we mutate the saddr, and we can release it right after.
Quoted 132 lines
net/ipv4/inet_hashtables.c +16
-7/* Allocate a bind2 bucket ahead of time to avoid permanently putting * the bhash2 table in an inconsistent state if a new tb2 bucket * allocation fails. */ new_tb2 =kmem_cache_alloc (hinfo->bind2_bucket_cachep,GFP_ATOMIC ); -if (!new_tb2) -return -ENOMEM ; +if (!new_tb2) { + err = -ENOMEM ; + goto unlock; + } head2 =inet_bhashfn_portaddr (hinfo, sk, net, port); -spin_lock_bh (&prev_saddr->lock); +spin_lock (&head2->lock);__sk_del_bind2_node (sk);inet_bind2_bucket_destroy (hinfo->bind2_bucket_cachep,inet_csk (sk)->icsk_bind2_hash); -spin_unlock_bh (&prev_saddr->lock); +spin_unlock (&head2->lock); + +inet_update_saddr (sk, saddr, family); -spin_lock_bh (&head2->lock); + head2 =inet_bhashfn_portaddr (hinfo, sk, net, port); + +spin_lock (&head2->lock); tb2 =inet_bind2_bucket_find (head2, net, port, l3mdev, sk);if (!tb2) { tb2 = new_tb2;@@ -890,12 +924,15 @@ int inet_bhash2_update_saddr(struct inet_bind_hashbucket *prev_saddr, struct soc }sk_add_bind2_node (sk, &tb2->owners);inet_csk (sk)->icsk_bind2_hash = tb2; -spin_unlock_bh (&head2->lock); +spin_unlock (&head2->lock);if (tb2 != new_tb2)kmem_cache_free (hinfo->bind2_bucket_cachep, new_tb2); -return 0 ; +unlock: +spin_unlock_bh (&head->lock); +out: +return err; }EXPORT_SYMBOL_GPL (inet_bhash2_update_saddr);net/ipv4/tcp_ipv4.c +4
-16@@ -199,15 +199,14 @@ static int tcp_v4_pre_connect(struct sock *sk, struct sockaddr *uaddr,/* This will initiate an outgoing connection. */ int tcp_v4_connect (struct sock *sk,struct sockaddr *uaddr,int addr_len) { -struct inet_bind_hashbucket *prev_addr_hashbucket =NULL ;struct sockaddr_in *usin = (struct sockaddr_in *)uaddr;struct inet_timewait_death_row *tcp_death_row; -__be32 daddr, nexthop, prev_sk_rcv_saddr;struct inet_sock *inet =inet_sk (sk);struct tcp_sock *tp =tcp_sk (sk);struct ip_options_rcu *inet_opt;struct net *net =sock_net (sk);__be16 orig_sport, orig_dport; +__be32 daddr, nexthop;struct flowi4 *fl4;struct rtable *rt;int err;@@ -251,24 +250,13 @@ int tcp_v4_connect(struct sock *sk, struct sockaddr *uaddr, int addr_len) tcp_death_row = &sock_net (sk)->ipv4.tcp_death_row;if (!inet->inet_saddr) { -if (inet_csk (sk)->icsk_bind2_hash) { - prev_addr_hashbucket =inet_bhashfn_portaddr (tcp_death_row->hashinfo, - sk, net, inet->inet_num); - prev_sk_rcv_saddr = sk->sk_rcv_saddr; - } - inet->inet_saddr = fl4->saddr; - } - -sk_rcv_saddr_set (sk, inet->inet_saddr); - -if (prev_addr_hashbucket) { - err =inet_bhash2_update_saddr (prev_addr_hashbucket, sk); + err =inet_bhash2_update_saddr (sk, &fl4->saddr,AF_INET );if (err) { - inet->inet_saddr =0 ; -sk_rcv_saddr_set (sk, prev_sk_rcv_saddr);ip_rt_put (rt);return err; } + }else { +sk_rcv_saddr_set (sk, inet->inet_saddr); }if (tp->rx_opt.ts_recent_stamp && inet->inet_daddr != daddr) {net/ipv6/tcp_ipv6.c +3
-17@@ -292,24 +292,11 @@ static int tcp_v6_connect(struct sock *sk, struct sockaddr *uaddr, tcp_death_row = &sock_net (sk)->ipv4.tcp_death_row;if (!saddr) { -struct inet_bind_hashbucket *prev_addr_hashbucket =NULL ; -struct in6_addr prev_v6_rcv_saddr; - -if (icsk->icsk_bind2_hash) { - prev_addr_hashbucket =inet_bhashfn_portaddr (tcp_death_row->hashinfo, - sk, net, inet->inet_num); - prev_v6_rcv_saddr = sk->sk_v6_rcv_saddr; - } saddr = &fl6.saddr; - sk->sk_v6_rcv_saddr = *saddr; -if (prev_addr_hashbucket) { - err =inet_bhash2_update_saddr (prev_addr_hashbucket, sk); -if (err) { - sk->sk_v6_rcv_saddr = prev_v6_rcv_saddr; - goto failure; - } - } + err =inet_bhash2_update_saddr (sk, saddr,AF_INET6 ); +if (err) + goto failure; }/* set the source address */ --2.30.2
Re: [PATCH v3 net 3/4] dccp/tcp: Update saddr under bhash's lock.
From: Joanne Koong <[email protected]> Date: Mon, 21 Nov 2022 14:52:56 -0800
Quoted 212 lines · Kuniyuki Iwashima
Quoted 211 lines
From: Joanne Koong <[email protected]> Date: Fri, 18 Nov 2022 15:20:20 -0800Quoted 207 lines · Kuniyuki Iwashima
Quoted 6 lines
When we call
connect()for a socket bound to a wildcard address, we update saddr locklessly. However, it could result in a data race; another thread iterating over bhash might see a corrupted address.Let's update saddr under the bhash bucket's lock.
Thanks for the quick turnaround!
Quoted 166 lines
Fixes 3df80d9320bc ("[DCCP]: Introduce DCCPv6")Fixes 7c657876b63c ("[DCCP]: Initial implementation")Fixes 1da177e4c3f4 ("Linux-2.6.12-rc2")Signed-off-by Kuniyuki Iwashima <[email protected]>--- include/net/inet_hashtables.h | 2 +- net/dccp/ipv4.c | 22 +++----------- net/dccp/ipv6.c | 23 +++------------ net/ipv4/af_inet.c | 11 +------ net/ipv4/inet_hashtables.c | 55 +++++++++++++++++++++++++++++------ net/ipv4/tcp_ipv4.c | 20 +++---------- net/ipv6/tcp_ipv6.c | 19 ++---------- 7 files changed, 63 insertions(+), 89 deletions(-)include/net/inet_hashtables.h +1
-1@@ -281,7 +281,7 @@ inet_bhash2_addr_any_hashbucket(const struct sock *sk, const struct net *net, in* sk_v6_rcv_saddr (ipv6) changes after it has been binded. The socket's * rcv_saddr field should already have been updated when this is called. */ -int inet_bhash2_update_saddr (struct inet_bind_hashbucket *prev_saddr,struct sock *sk); +int inet_bhash2_update_saddr (struct sock *sk,void *saddr,int family);void inet_bind_hash (struct sock *sk,struct inet_bind_bucket *tb,struct inet_bind2_bucket *tb2,unsigned short port);net/dccp/ipv4.c +4
-17@@ -45,11 +45,10 @@ static unsigned int dccp_v4_pernet_id __read_mostly;int dccp_v4_connect (struct sock *sk,struct sockaddr *uaddr,int addr_len) {const struct sockaddr_in *usin = (struct sockaddr_in *)uaddr; -struct inet_bind_hashbucket *prev_addr_hashbucket =NULL ; -__be32 daddr, nexthop, prev_sk_rcv_saddr;struct inet_sock *inet =inet_sk (sk);struct dccp_sock *dp =dccp_sk (sk);__be16 orig_sport, orig_dport; +__be32 daddr, nexthop;struct flowi4 *fl4;struct rtable *rt;int err;@@ -91,26 +90,13 @@ int dccp_v4_connect(struct sock *sk, struct sockaddr *uaddr, int addr_len) daddr = fl4->daddr;if (inet->inet_saddr =0 ) { -if (inet_csk (sk)->icsk_bind2_hash) { - prev_addr_hashbucket > > > > - inet_bhashfn_portaddr(&dccp_hashinfo, sk, -sock_net (sk), - inet->inet_num); - prev_sk_rcv_saddr = sk->sk_rcv_saddr; - } - inet->inet_saddr = fl4->saddr; - } - -sk_rcv_saddr_set (sk, inet->inet_saddr); - -if (prev_addr_hashbucket) { - err =inet_bhash2_update_saddr (prev_addr_hashbucket, sk); + err =inet_bhash2_update_saddr (sk, &fl4->saddr,AF_INET );if (err) { - inet->inet_saddr =0 ; -sk_rcv_saddr_set (sk, prev_sk_rcv_saddr);ip_rt_put (rt);return err; } + }else { +sk_rcv_saddr_set (sk, inet->inet_saddr); } inet->inet_dport = usin->sin_port;net/dccp/ipv6.c +4
-19@@ -934,26 +934,11 @@ static int dccp_v6_connect(struct sock *sk, struct sockaddr *uaddr, }if (saddr =NULL ) { -struct inet_bind_hashbucket *prev_addr_hashbucket =NULL ; -struct in6_addr prev_v6_rcv_saddr; - -if (icsk->icsk_bind2_hash) { - prev_addr_hashbucket =inet_bhashfn_portaddr (&dccp_hashinfo, - sk,sock_net (sk), - inet->inet_num); - prev_v6_rcv_saddr = sk->sk_v6_rcv_saddr; - } - saddr = &fl6.saddr; - sk->sk_v6_rcv_saddr = *saddr; - -if (prev_addr_hashbucket) { - err =inet_bhash2_update_saddr (prev_addr_hashbucket, sk); -if (err) { - sk->sk_v6_rcv_saddr = prev_v6_rcv_saddr; - goto failure; - } - } + + err =inet_bhash2_update_saddr (sk, saddr,AF_INET6 ); +if (err) + goto failure; }/* set the source address */ net/ipv4/af_inet.c +1
-9@@ -1230,7 +1230,6 @@ EXPORT_SYMBOL(inet_unregister_protosw);static int inet_sk_reselect_saddr (struct sock *sk) { -struct inet_bind_hashbucket *prev_addr_hashbucket;struct inet_sock *inet =inet_sk (sk);__be32 old_saddr = inet->inet_saddr;__be32 daddr = inet->inet_daddr;@@ -1260,16 +1259,8 @@ static int inet_sk_reselect_saddr(struct sock *sk)return 0 ; } - prev_addr_hashbucket > > > > - inet_bhashfn_portaddr(tcp_or_dccp_get_hashinfo(sk), sk, -sock_net (sk), inet->inet_num); - - inet->inet_saddr = inet->inet_rcv_saddr = new_saddr; - - err =inet_bhash2_update_saddr (prev_addr_hashbucket, sk); + err =inet_bhash2_update_saddr (sk, &new_saddr,AF_INET );if (err) { - inet->inet_saddr = old_saddr; - inet->inet_rcv_saddr = old_saddr;ip_rt_put (rt);return err; }net/ipv4/inet_hashtables.c +15
-1@@ -858,31 +858,65 @@ inet_bhash2_addr_any_hashbucket(const struct sock *sk, const struct net *net, inreturn &hinfo->bhash2[hash & (hinfo->bhash_size -1 )]; } -int inet_bhash2_update_saddr (struct inet_bind_hashbucket *prev_saddr,struct sock *sk) +static void inet_update_saddr (struct sock *sk,void *saddr,int family) +{ +if (family =AF_INET ) { +inet_sk (sk)->inet_saddr = *(__be32 *)saddr; +sk_rcv_saddr_set (sk,inet_sk (sk)->inet_saddr); + } +#if IS_ENABLED (CONFIG_IPV6 ) + else { + sk->sk_v6_rcv_saddr = *(struct in6_addr *)saddr; + } +#endif +} + +int inet_bhash2_update_saddr (struct sock *sk,void *saddr,int family) {struct inet_hashinfo *hinfo =tcp_or_dccp_get_hashinfo (sk); +struct inet_bind_hashbucket *head, *head2;struct inet_bind2_bucket *tb2, *new_tb2;int l3mdev =inet_sk_bound_l3mdev (sk);net/ipv4/inet_hashtables.c inet_bhash2_addr_any_hashbucket(const struct sock *sk, const struct net *net, in · line 888
Quoted 10 lines
net/ipv4/inet_hashtables.c +7
-1-struct inet_bind_hashbucket *head2;int port =inet_sk (sk)->inet_num;struct net *net =sock_net (sk); +int bhash, err =0 ; + +if (!inet_csk (sk)->icsk_bind2_hash) { +/* Not bind()ed before. */ +inet_update_saddr (sk, saddr, family); + goto out; + }I think it would be cleaner if this logic were outside
bhash2_update_saddr(), since this mutates the sk's address when the socket hasn't been previously bound to bhash2. I think something like this would be clearer:static int
inet_update_saddr(struct sock *sk, void *saddr, int family){ if (!inet_csk(sk)->icsk_bind2_hash) { update_sk_saddr(sk, saddr, family) return 0; } return inet_bhash2_update_saddr(sk, saddr, family); }and then from dccp/tcp_ipv4/6_connect(), we just call
inet_update_saddr(). This also "moves" the lower-level implementation details (eg underlying bind tables) toinet_hashtables.c, instead of it being mentioned in the higher dccp_tcp_ipv4/6 layers.What are your thoughts?
Quoted 7 lines
Quoted 2 lines
Hmm.. There are other users of
inet_reset_saddr(). If we hide the details of reset & bhash2 logic too, we need a cleanup patch below.I don't think reset needs to be hidden / abstracted away, that sounds generic. But bhash2 sounds like an implementation detail, which is why it seems cleaner to me if we didn't have bhash2 mentioned in the dccp/tcp_ipv4/6 layers.
Sorry, my description was not clear.
Now we have inet_bhash2_update_saddr() and inet_bhash2_reset_saddr().
If we hide bhash2 logic as inet_update_saddr(),
- inet_update_saddr
- if (...)
- __inet_update_saddr
- inet_bhash2_update_saddr
- __inet_update_saddr
then we will change inet_bhash2_reset_saddr() logic like
- inet_reset_saddr
- if (...)
- __inet_reset_saddr
- inet_bhash2_reset_saddr
- __inet_reset_saddr
inet_update_saddr() is what we newly added, but inet_reset_saddr() is not. There are existing users of inet_reset_saddr(), e.g. UDP, and we need a patch below.
If we don't clean up, other protocol not using bhash2 also go through this condition via inet_reset_saddr(), which is originally unnecessary.
if (!inet_csk(sk)->icsk_bind2_hash)
What do you think ?
Quoted 238 lines
Quoted 4 lines
So, I'll keep this part as is in the next spin. If needed, I can post a followup for net-next.
And I noticed UDP has the same problem, this will be another series.
Sounds good, looking forward to your UDP patchset.
Quoted 231 lines
Thank you!
---8<---
include/net/inet_hashtables.h +1
-0@@ -277,6 +277,7 @@ inet_bhashfn_portaddr(const struct inet_hashinfo *hinfo, const struct sock *sk,struct inet_bind_hashbucket *inet_bhash2_addr_any_hashbucket (const struct sock *sk,const struct net *net,int port); +void inet_reset_saddr (struct sock *sk);int inet_update_saddr (struct sock *sk,void *saddr,int family); void inet_bind_hash(struct sock *sk, struct inet_bind_bucket *tb,include/net/ip.h +1
-1@@ -632,7 +632,7 @@ static inline void ip_ipgre_mc_map(__be32 naddr, const unsigned char *broadcast,#include <linux/ipv6.h> #endif -static __inline__void inet_reset_saddr (struct sock *sk) +static __inline__void __inet_reset_saddr (struct sock *sk) {inet_sk (sk)->inet_rcv_saddr =inet_sk (sk)->inet_saddr =0 ;#if IS_ENABLED (CONFIG_IPV6 )net/ipv4/inet_hashtables.c +6
-0@@ -941,6 +941,12 @@ int inet_update_saddr(struct sock *sk, void *saddr, int family) }EXPORT_SYMBOL_GPL (inet_update_saddr); +void inet_reset_saddr (struct sock *sk) +{ +__inet_reset_saddr (sk); +} +EXPORT_SYMBOL_GPL (inet_reset_saddr); +/* RFC 6056 3.3.4. Algorithm 4: Double-Hash Port Selection Algorithm * Note that we use 32bit integers (vs RFC 'short integers') * because 2^16 is not a multiple of num_ephemeral and this net/ipv4/udp.c +1
-1@@ -1971,7 +1971,7 @@ int __udp_disconnect(struct sock *sk, int flags)sock_rps_reset_rxhash (sk); sk->sk_bound_dev_if =0 ;if (!(sk->sk_userlocks &SOCK_BINDADDR_LOCK )) { -inet_reset_saddr (sk); +__inet_reset_saddr (sk);if (sk->sk_prot->rehash && (sk->sk_userlocks &SOCK_BINDPORT_LOCK )) sk->sk_prot->rehash (sk);net/ipv6/af_inet6.c +2
-3@@ -411,7 +411,7 @@ static int __inet6_bind(struct sock *sk, struct sockaddr *uaddr, int addr_len, (flags & BIND_FORCE_ADDRESS_NO_PORT))) {if (sk->sk_prot->get_port (sk, snum)) { sk->sk_ipv6only = saved_ipv6only; -inet_reset_saddr (sk); +__inet_reset_saddr (sk); err = -EADDRINUSE ; goto out; }@@ -419,7 +419,7 @@ static int __inet6_bind(struct sock *sk, struct sockaddr *uaddr, int addr_len, err =BPF_CGROUP_RUN_PROG_INET6_POST_BIND (sk);if (err) { sk->sk_ipv6only = saved_ipv6only; -inet_reset_saddr (sk); +__inet_reset_saddr (sk);if (sk->sk_prot->put_port) sk->sk_prot->put_port (sk); goto out; ---8<---Quoted 148 lines
net/ipv4/inet_hashtables.c inet_bhash2_addr_any_hashbucket(const struct sock *sk, const struct net *net, in · line 896
Quoted 11 lines
net/ipv4/inet_hashtables.c +8
-0+ + bhash =inet_bhashfn (net, port, hinfo->bhash_size); + head = &hinfo->bhash[bhash]; + +/* If we change saddr locklessly, another thread +* iterating over bhash might see corrupted address. +*/ +spin_lock_bh (&head->lock);I don't think we should be acquiring the bhash lock here. I think we only need to acquire it right before we mutate the saddr, and we can release it right after.
Quoted 132 lines
net/ipv4/inet_hashtables.c +16
-7/* Allocate a bind2 bucket ahead of time to avoid permanently putting * the bhash2 table in an inconsistent state if a new tb2 bucket * allocation fails. */ new_tb2 =kmem_cache_alloc (hinfo->bind2_bucket_cachep,GFP_ATOMIC ); -if (!new_tb2) -return -ENOMEM ; +if (!new_tb2) { + err = -ENOMEM ; + goto unlock; + } head2 =inet_bhashfn_portaddr (hinfo, sk, net, port); -spin_lock_bh (&prev_saddr->lock); +spin_lock (&head2->lock);__sk_del_bind2_node (sk);inet_bind2_bucket_destroy (hinfo->bind2_bucket_cachep,inet_csk (sk)->icsk_bind2_hash); -spin_unlock_bh (&prev_saddr->lock); +spin_unlock (&head2->lock); + +inet_update_saddr (sk, saddr, family); -spin_lock_bh (&head2->lock); + head2 =inet_bhashfn_portaddr (hinfo, sk, net, port); + +spin_lock (&head2->lock); tb2 =inet_bind2_bucket_find (head2, net, port, l3mdev, sk);if (!tb2) { tb2 = new_tb2;@@ -890,12 +924,15 @@ int inet_bhash2_update_saddr(struct inet_bind_hashbucket *prev_saddr, struct soc }sk_add_bind2_node (sk, &tb2->owners);inet_csk (sk)->icsk_bind2_hash = tb2; -spin_unlock_bh (&head2->lock); +spin_unlock (&head2->lock);if (tb2 != new_tb2)kmem_cache_free (hinfo->bind2_bucket_cachep, new_tb2); -return 0 ; +unlock: +spin_unlock_bh (&head->lock); +out: +return err; }EXPORT_SYMBOL_GPL (inet_bhash2_update_saddr);net/ipv4/tcp_ipv4.c +4
-16@@ -199,15 +199,14 @@ static int tcp_v4_pre_connect(struct sock *sk, struct sockaddr *uaddr,/* This will initiate an outgoing connection. */ int tcp_v4_connect (struct sock *sk,struct sockaddr *uaddr,int addr_len) { -struct inet_bind_hashbucket *prev_addr_hashbucket =NULL ;struct sockaddr_in *usin = (struct sockaddr_in *)uaddr;struct inet_timewait_death_row *tcp_death_row; -__be32 daddr, nexthop, prev_sk_rcv_saddr;struct inet_sock *inet =inet_sk (sk);struct tcp_sock *tp =tcp_sk (sk);struct ip_options_rcu *inet_opt;struct net *net =sock_net (sk);__be16 orig_sport, orig_dport; +__be32 daddr, nexthop;struct flowi4 *fl4;struct rtable *rt;int err;@@ -251,24 +250,13 @@ int tcp_v4_connect(struct sock *sk, struct sockaddr *uaddr, int addr_len) tcp_death_row = &sock_net (sk)->ipv4.tcp_death_row;if (!inet->inet_saddr) { -if (inet_csk (sk)->icsk_bind2_hash) { - prev_addr_hashbucket =inet_bhashfn_portaddr (tcp_death_row->hashinfo, - sk, net, inet->inet_num); - prev_sk_rcv_saddr = sk->sk_rcv_saddr; - } - inet->inet_saddr = fl4->saddr; - } - -sk_rcv_saddr_set (sk, inet->inet_saddr); - -if (prev_addr_hashbucket) { - err =inet_bhash2_update_saddr (prev_addr_hashbucket, sk); + err =inet_bhash2_update_saddr (sk, &fl4->saddr,AF_INET );if (err) { - inet->inet_saddr =0 ; -sk_rcv_saddr_set (sk, prev_sk_rcv_saddr);ip_rt_put (rt);return err; } + }else { +sk_rcv_saddr_set (sk, inet->inet_saddr); }if (tp->rx_opt.ts_recent_stamp && inet->inet_daddr != daddr) {net/ipv6/tcp_ipv6.c +3
-17@@ -292,24 +292,11 @@ static int tcp_v6_connect(struct sock *sk, struct sockaddr *uaddr, tcp_death_row = &sock_net (sk)->ipv4.tcp_death_row;if (!saddr) { -struct inet_bind_hashbucket *prev_addr_hashbucket =NULL ; -struct in6_addr prev_v6_rcv_saddr; - -if (icsk->icsk_bind2_hash) { - prev_addr_hashbucket =inet_bhashfn_portaddr (tcp_death_row->hashinfo, - sk, net, inet->inet_num); - prev_v6_rcv_saddr = sk->sk_v6_rcv_saddr; - } saddr = &fl6.saddr; - sk->sk_v6_rcv_saddr = *saddr; -if (prev_addr_hashbucket) { - err =inet_bhash2_update_saddr (prev_addr_hashbucket, sk); -if (err) { - sk->sk_v6_rcv_saddr = prev_v6_rcv_saddr; - goto failure; - } - } + err =inet_bhash2_update_saddr (sk, saddr,AF_INET6 ); +if (err) + goto failure; }/* set the source address */ --2.30.2
Re: [PATCH v3 net 3/4] dccp/tcp: Update saddr under bhash's lock.
Quoted 247 lines · Kuniyuki Iwashima
From: Joanne Koong <[email protected]> Date: Mon, 21 Nov 2022 14:52:56 -0800Quoted 212 lines · Kuniyuki Iwashima
Quoted 211 lines
From: Joanne Koong <[email protected]> Date: Fri, 18 Nov 2022 15:20:20 -0800Quoted 207 lines · Kuniyuki Iwashima
Quoted 6 lines
When we call
connect()for a socket bound to a wildcard address, we update saddr locklessly. However, it could result in a data race; another thread iterating over bhash might see a corrupted address.Let's update saddr under the bhash bucket's lock.
Thanks for the quick turnaround!
Quoted 166 lines
Fixes 3df80d9320bc ("[DCCP]: Introduce DCCPv6")Fixes 7c657876b63c ("[DCCP]: Initial implementation")Fixes 1da177e4c3f4 ("Linux-2.6.12-rc2")Signed-off-by Kuniyuki Iwashima <[email protected]>--- include/net/inet_hashtables.h | 2 +- net/dccp/ipv4.c | 22 +++----------- net/dccp/ipv6.c | 23 +++------------ net/ipv4/af_inet.c | 11 +------ net/ipv4/inet_hashtables.c | 55 +++++++++++++++++++++++++++++------ net/ipv4/tcp_ipv4.c | 20 +++---------- net/ipv6/tcp_ipv6.c | 19 ++---------- 7 files changed, 63 insertions(+), 89 deletions(-)include/net/inet_hashtables.h +1
-1@@ -281,7 +281,7 @@ inet_bhash2_addr_any_hashbucket(const struct sock *sk, const struct net *net, in* sk_v6_rcv_saddr (ipv6) changes after it has been binded. The socket's * rcv_saddr field should already have been updated when this is called. */ -int inet_bhash2_update_saddr (struct inet_bind_hashbucket *prev_saddr,struct sock *sk); +int inet_bhash2_update_saddr (struct sock *sk,void *saddr,int family);void inet_bind_hash (struct sock *sk,struct inet_bind_bucket *tb,struct inet_bind2_bucket *tb2,unsigned short port);net/dccp/ipv4.c +4
-17@@ -45,11 +45,10 @@ static unsigned int dccp_v4_pernet_id __read_mostly;int dccp_v4_connect (struct sock *sk,struct sockaddr *uaddr,int addr_len) {const struct sockaddr_in *usin = (struct sockaddr_in *)uaddr; -struct inet_bind_hashbucket *prev_addr_hashbucket =NULL ; -__be32 daddr, nexthop, prev_sk_rcv_saddr;struct inet_sock *inet =inet_sk (sk);struct dccp_sock *dp =dccp_sk (sk);__be16 orig_sport, orig_dport; +__be32 daddr, nexthop;struct flowi4 *fl4;struct rtable *rt;int err;@@ -91,26 +90,13 @@ int dccp_v4_connect(struct sock *sk, struct sockaddr *uaddr, int addr_len) daddr = fl4->daddr;if (inet->inet_saddr =0 ) { -if (inet_csk (sk)->icsk_bind2_hash) { - prev_addr_hashbucket > > > > > - inet_bhashfn_portaddr(&dccp_hashinfo, sk, -sock_net (sk), - inet->inet_num); - prev_sk_rcv_saddr = sk->sk_rcv_saddr; - } - inet->inet_saddr = fl4->saddr; - } - -sk_rcv_saddr_set (sk, inet->inet_saddr); - -if (prev_addr_hashbucket) { - err =inet_bhash2_update_saddr (prev_addr_hashbucket, sk); + err =inet_bhash2_update_saddr (sk, &fl4->saddr,AF_INET );if (err) { - inet->inet_saddr =0 ; -sk_rcv_saddr_set (sk, prev_sk_rcv_saddr);ip_rt_put (rt);return err; } + }else { +sk_rcv_saddr_set (sk, inet->inet_saddr); } inet->inet_dport = usin->sin_port;net/dccp/ipv6.c +4
-19@@ -934,26 +934,11 @@ static int dccp_v6_connect(struct sock *sk, struct sockaddr *uaddr, }if (saddr =NULL ) { -struct inet_bind_hashbucket *prev_addr_hashbucket =NULL ; -struct in6_addr prev_v6_rcv_saddr; - -if (icsk->icsk_bind2_hash) { - prev_addr_hashbucket =inet_bhashfn_portaddr (&dccp_hashinfo, - sk,sock_net (sk), - inet->inet_num); - prev_v6_rcv_saddr = sk->sk_v6_rcv_saddr; - } - saddr = &fl6.saddr; - sk->sk_v6_rcv_saddr = *saddr; - -if (prev_addr_hashbucket) { - err =inet_bhash2_update_saddr (prev_addr_hashbucket, sk); -if (err) { - sk->sk_v6_rcv_saddr = prev_v6_rcv_saddr; - goto failure; - } - } + + err =inet_bhash2_update_saddr (sk, saddr,AF_INET6 ); +if (err) + goto failure; }/* set the source address */ net/ipv4/af_inet.c +1
-9@@ -1230,7 +1230,6 @@ EXPORT_SYMBOL(inet_unregister_protosw);static int inet_sk_reselect_saddr (struct sock *sk) { -struct inet_bind_hashbucket *prev_addr_hashbucket;struct inet_sock *inet =inet_sk (sk);__be32 old_saddr = inet->inet_saddr;__be32 daddr = inet->inet_daddr;@@ -1260,16 +1259,8 @@ static int inet_sk_reselect_saddr(struct sock *sk)return 0 ; } - prev_addr_hashbucket > > > > > - inet_bhashfn_portaddr(tcp_or_dccp_get_hashinfo(sk), sk, -sock_net (sk), inet->inet_num); - - inet->inet_saddr = inet->inet_rcv_saddr = new_saddr; - - err =inet_bhash2_update_saddr (prev_addr_hashbucket, sk); + err =inet_bhash2_update_saddr (sk, &new_saddr,AF_INET );if (err) { - inet->inet_saddr = old_saddr; - inet->inet_rcv_saddr = old_saddr;ip_rt_put (rt);return err; }net/ipv4/inet_hashtables.c +15
-1@@ -858,31 +858,65 @@ inet_bhash2_addr_any_hashbucket(const struct sock *sk, const struct net *net, inreturn &hinfo->bhash2[hash & (hinfo->bhash_size -1 )]; } -int inet_bhash2_update_saddr (struct inet_bind_hashbucket *prev_saddr,struct sock *sk) +static void inet_update_saddr (struct sock *sk,void *saddr,int family) +{ +if (family =AF_INET ) { +inet_sk (sk)->inet_saddr = *(__be32 *)saddr; +sk_rcv_saddr_set (sk,inet_sk (sk)->inet_saddr); + } +#if IS_ENABLED (CONFIG_IPV6 ) + else { + sk->sk_v6_rcv_saddr = *(struct in6_addr *)saddr; + } +#endif +} + +int inet_bhash2_update_saddr (struct sock *sk,void *saddr,int family) {struct inet_hashinfo *hinfo =tcp_or_dccp_get_hashinfo (sk); +struct inet_bind_hashbucket *head, *head2;struct inet_bind2_bucket *tb2, *new_tb2;int l3mdev =inet_sk_bound_l3mdev (sk);net/ipv4/inet_hashtables.c inet_bhash2_addr_any_hashbucket(const struct sock *sk, const struct net *net, in · line 888
Quoted 10 lines
net/ipv4/inet_hashtables.c +7
-1-struct inet_bind_hashbucket *head2;int port =inet_sk (sk)->inet_num;struct net *net =sock_net (sk); +int bhash, err =0 ; + +if (!inet_csk (sk)->icsk_bind2_hash) { +/* Not bind()ed before. */ +inet_update_saddr (sk, saddr, family); + goto out; + }I think it would be cleaner if this logic were outside
bhash2_update_saddr(), since this mutates the sk's address when the socket hasn't been previously bound to bhash2. I think something like this would be clearer:static int
inet_update_saddr(struct sock *sk, void *saddr, int family){ if (!inet_csk(sk)->icsk_bind2_hash) { update_sk_saddr(sk, saddr, family) return 0; } return inet_bhash2_update_saddr(sk, saddr, family); }and then from dccp/tcp_ipv4/6_connect(), we just call
inet_update_saddr(). This also "moves" the lower-level implementation details (eg underlying bind tables) toinet_hashtables.c, instead of it being mentioned in the higher dccp_tcp_ipv4/6 layers.What are your thoughts?
Quoted 7 lines
Quoted 2 lines
Hmm.. There are other users of
inet_reset_saddr(). If we hide the details of reset & bhash2 logic too, we need a cleanup patch below.I don't think reset needs to be hidden / abstracted away, that sounds generic. But bhash2 sounds like an implementation detail, which is why it seems cleaner to me if we didn't have bhash2 mentioned in the dccp/tcp_ipv4/6 layers.
Sorry, my description was not clear.
Now we have
inet_bhash2_update_saddr()andinet_bhash2_reset_saddr().If we hide bhash2 logic as
inet_update_saddr(),-
inet_update_saddr
- if (...)
-__inet_update_saddr
-inet_bhash2_update_saddr
-__inet_update_saddrthen we will change
inet_bhash2_reset_saddr()logic like-
inet_reset_saddr
- if (...)
-__inet_reset_saddr
-inet_bhash2_reset_saddr
-__inet_reset_saddr
inet_update_saddr()is what we newly added, butinet_reset_saddr()is not. There are existing users ofinet_reset_saddr(), e.g. UDP, and we need a patch below.
Quoted 7 lines
If we don't clean up, other protocol not using bhash2 also go through this condition via
inet_reset_saddr(), which is originally unnecessary.if (!inet_csk(sk)->icsk_bind2_hash)
What do you think ?
Thanks for clarifying! I think this is a minor detail we can revisit in the future if need be, and that we should just go with what you have in this patch.
Quoted 239 lines
Quoted 239 lines
Quoted 5 lines
So, I'll keep this part as is in the next spin. If needed, I can post a followup for net-next.
And I noticed UDP has the same problem, this will be another series.
Sounds good, looking forward to your UDP patchset.
Quoted 231 lines
Thank you!
---8<---
include/net/inet_hashtables.h +1
-0@@ -277,6 +277,7 @@ inet_bhashfn_portaddr(const struct inet_hashinfo *hinfo, const struct sock *sk,struct inet_bind_hashbucket *inet_bhash2_addr_any_hashbucket (const struct sock *sk,const struct net *net,int port); +void inet_reset_saddr (struct sock *sk);int inet_update_saddr (struct sock *sk,void *saddr,int family); void inet_bind_hash(struct sock *sk, struct inet_bind_bucket *tb,include/net/ip.h +1
-1@@ -632,7 +632,7 @@ static inline void ip_ipgre_mc_map(__be32 naddr, const unsigned char *broadcast,#include <linux/ipv6.h> #endif -static __inline__void inet_reset_saddr (struct sock *sk) +static __inline__void __inet_reset_saddr (struct sock *sk) {inet_sk (sk)->inet_rcv_saddr =inet_sk (sk)->inet_saddr =0 ;#if IS_ENABLED (CONFIG_IPV6 )net/ipv4/inet_hashtables.c +6
-0@@ -941,6 +941,12 @@ int inet_update_saddr(struct sock *sk, void *saddr, int family) }EXPORT_SYMBOL_GPL (inet_update_saddr); +void inet_reset_saddr (struct sock *sk) +{ +__inet_reset_saddr (sk); +} +EXPORT_SYMBOL_GPL (inet_reset_saddr); +/* RFC 6056 3.3.4. Algorithm 4: Double-Hash Port Selection Algorithm * Note that we use 32bit integers (vs RFC 'short integers') * because 2^16 is not a multiple of num_ephemeral and this net/ipv4/udp.c +1
-1@@ -1971,7 +1971,7 @@ int __udp_disconnect(struct sock *sk, int flags)sock_rps_reset_rxhash (sk); sk->sk_bound_dev_if =0 ;if (!(sk->sk_userlocks &SOCK_BINDADDR_LOCK )) { -inet_reset_saddr (sk); +__inet_reset_saddr (sk);if (sk->sk_prot->rehash && (sk->sk_userlocks &SOCK_BINDPORT_LOCK )) sk->sk_prot->rehash (sk);net/ipv6/af_inet6.c +2
-3@@ -411,7 +411,7 @@ static int __inet6_bind(struct sock *sk, struct sockaddr *uaddr, int addr_len, (flags & BIND_FORCE_ADDRESS_NO_PORT))) {if (sk->sk_prot->get_port (sk, snum)) { sk->sk_ipv6only = saved_ipv6only; -inet_reset_saddr (sk); +__inet_reset_saddr (sk); err = -EADDRINUSE ; goto out; }@@ -419,7 +419,7 @@ static int __inet6_bind(struct sock *sk, struct sockaddr *uaddr, int addr_len, err =BPF_CGROUP_RUN_PROG_INET6_POST_BIND (sk);if (err) { sk->sk_ipv6only = saved_ipv6only; -inet_reset_saddr (sk); +__inet_reset_saddr (sk);if (sk->sk_prot->put_port) sk->sk_prot->put_port (sk); goto out; ---8<---Quoted 148 lines
net/ipv4/inet_hashtables.c inet_bhash2_addr_any_hashbucket(const struct sock *sk, const struct net *net, in · line 896
Quoted 11 lines
net/ipv4/inet_hashtables.c +8
-0+ + bhash =inet_bhashfn (net, port, hinfo->bhash_size); + head = &hinfo->bhash[bhash]; + +/* If we change saddr locklessly, another thread +* iterating over bhash might see corrupted address. +*/ +spin_lock_bh (&head->lock);I don't think we should be acquiring the bhash lock here. I think we only need to acquire it right before we mutate the saddr, and we can release it right after.
Quoted 132 lines
net/ipv4/inet_hashtables.c +16
-7/* Allocate a bind2 bucket ahead of time to avoid permanently putting * the bhash2 table in an inconsistent state if a new tb2 bucket * allocation fails. */ new_tb2 =kmem_cache_alloc (hinfo->bind2_bucket_cachep,GFP_ATOMIC ); -if (!new_tb2) -return -ENOMEM ; +if (!new_tb2) { + err = -ENOMEM ; + goto unlock; + } head2 =inet_bhashfn_portaddr (hinfo, sk, net, port); -spin_lock_bh (&prev_saddr->lock); +spin_lock (&head2->lock);__sk_del_bind2_node (sk);inet_bind2_bucket_destroy (hinfo->bind2_bucket_cachep,inet_csk (sk)->icsk_bind2_hash); -spin_unlock_bh (&prev_saddr->lock); +spin_unlock (&head2->lock); + +inet_update_saddr (sk, saddr, family); -spin_lock_bh (&head2->lock); + head2 =inet_bhashfn_portaddr (hinfo, sk, net, port); + +spin_lock (&head2->lock); tb2 =inet_bind2_bucket_find (head2, net, port, l3mdev, sk);if (!tb2) { tb2 = new_tb2;@@ -890,12 +924,15 @@ int inet_bhash2_update_saddr(struct inet_bind_hashbucket *prev_saddr, struct soc }sk_add_bind2_node (sk, &tb2->owners);inet_csk (sk)->icsk_bind2_hash = tb2; -spin_unlock_bh (&head2->lock); +spin_unlock (&head2->lock);if (tb2 != new_tb2)kmem_cache_free (hinfo->bind2_bucket_cachep, new_tb2); -return 0 ; +unlock: +spin_unlock_bh (&head->lock); +out: +return err; }EXPORT_SYMBOL_GPL (inet_bhash2_update_saddr);net/ipv4/tcp_ipv4.c +4
-16@@ -199,15 +199,14 @@ static int tcp_v4_pre_connect(struct sock *sk, struct sockaddr *uaddr,/* This will initiate an outgoing connection. */ int tcp_v4_connect (struct sock *sk,struct sockaddr *uaddr,int addr_len) { -struct inet_bind_hashbucket *prev_addr_hashbucket =NULL ;struct sockaddr_in *usin = (struct sockaddr_in *)uaddr;struct inet_timewait_death_row *tcp_death_row; -__be32 daddr, nexthop, prev_sk_rcv_saddr;struct inet_sock *inet =inet_sk (sk);struct tcp_sock *tp =tcp_sk (sk);struct ip_options_rcu *inet_opt;struct net *net =sock_net (sk);__be16 orig_sport, orig_dport; +__be32 daddr, nexthop;struct flowi4 *fl4;struct rtable *rt;int err;@@ -251,24 +250,13 @@ int tcp_v4_connect(struct sock *sk, struct sockaddr *uaddr, int addr_len) tcp_death_row = &sock_net (sk)->ipv4.tcp_death_row;if (!inet->inet_saddr) { -if (inet_csk (sk)->icsk_bind2_hash) { - prev_addr_hashbucket =inet_bhashfn_portaddr (tcp_death_row->hashinfo, - sk, net, inet->inet_num); - prev_sk_rcv_saddr = sk->sk_rcv_saddr; - } - inet->inet_saddr = fl4->saddr; - } - -sk_rcv_saddr_set (sk, inet->inet_saddr); - -if (prev_addr_hashbucket) { - err =inet_bhash2_update_saddr (prev_addr_hashbucket, sk); + err =inet_bhash2_update_saddr (sk, &fl4->saddr,AF_INET );if (err) { - inet->inet_saddr =0 ; -sk_rcv_saddr_set (sk, prev_sk_rcv_saddr);ip_rt_put (rt);return err; } + }else { +sk_rcv_saddr_set (sk, inet->inet_saddr); }if (tp->rx_opt.ts_recent_stamp && inet->inet_daddr != daddr) {net/ipv6/tcp_ipv6.c +3
-17@@ -292,24 +292,11 @@ static int tcp_v6_connect(struct sock *sk, struct sockaddr *uaddr, tcp_death_row = &sock_net (sk)->ipv4.tcp_death_row;if (!saddr) { -struct inet_bind_hashbucket *prev_addr_hashbucket =NULL ; -struct in6_addr prev_v6_rcv_saddr; - -if (icsk->icsk_bind2_hash) { - prev_addr_hashbucket =inet_bhashfn_portaddr (tcp_death_row->hashinfo, - sk, net, inet->inet_num); - prev_v6_rcv_saddr = sk->sk_v6_rcv_saddr; - } saddr = &fl6.saddr; - sk->sk_v6_rcv_saddr = *saddr; -if (prev_addr_hashbucket) { - err =inet_bhash2_update_saddr (prev_addr_hashbucket, sk); -if (err) { - sk->sk_v6_rcv_saddr = prev_v6_rcv_saddr; - goto failure; - } - } + err =inet_bhash2_update_saddr (sk, saddr,AF_INET6 ); +if (err) + goto failure; }/* set the source address */ --2.30.2
PATCH v3 4/4 dccp/tcp: Fixup bhash2 bucket when connect() fails.
In reply to Kuniyuki IwashimaIf a socket bound to a wildcard address fails to connect(), we only reset saddr and keep the port. Then, we have to fix up the bhash2 bucket; otherwise, the bucket has an inconsistent address in the list.
Also, listen() for such a socket will fire the WARN_ON() in inet_csk_get_port(). [0]
Note that when a system runs out of memory, we give up fixing the bucket and unlink sk from bhash and bhash2 by inet_put_port().
[0]:
WARNING: CPU: 0 PID: 207 at net/ipv4/inet_connection_sock.c:548 inet_csk_get_port (net/ipv4/inet_connection_sock.c:548 (discriminator 1)) Modules linked in: CPU: 0 PID: 207 Comm: bhash2_prev_rep Not tainted 6.1.0-rc3-00799-gc8421681c845 #63 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.0-1.amzn2022.0.1 04/01/2014 RIP: 0010:inet_csk_get_port (net/ipv4/inet_connection_sock.c:548 (discriminator 1)) Code: 74 a7 eb 93 48 8b 54 24 18 0f b7 cb 4c 89 e6 4c 89 ff e8 48 b2 ff ff 49 8b 87 18 04 00 00 e9 32 ff ff ff 0f 0b e9 34 ff ff ff <0f> 0b e9 42 ff ff ff 41 8b 7f 50 41 8b 4f 54 89 fe 81 f6 00 00 ff RSP: 0018:ffffc900003d7e50 EFLAGS: 00010202 RAX: ffff8881047fb500 RBX: 0000000000004e20 RCX: 0000000000000000 RDX: 000000000000000a RSI: 00000000fffffe00 RDI: 00000000ffffffff RBP: ffffffff8324dc00 R08: 0000000000000001 R09: 0000000000000001 R10: 0000000000000001 R11: 0000000000000001 R12: 0000000000000000 R13: 0000000000000001 R14: 0000000000004e20 R15: ffff8881054e1280 FS: 00007f8ac04dc740(0000) GS:ffff88842fc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000020001540 CR3: 00000001055fa003 CR4: 0000000000770ef0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 PKRU: 55555554 Call Trace: <TASK> inet_csk_listen_start (net/ipv4/inet_connection_sock.c:1205) inet_listen (net/ipv4/af_inet.c:228) __sys_listen (net/socket.c:1810) __x64_sys_listen (net/socket.c:1819 net/socket.c:1817 net/socket.c:1817) do_syscall_64 (arch/x86/entry/common.c:50 arch/x86/entry/common.c:80) entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:120) RIP: 0033:0x7f8ac051de5d Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 93 af 1b 00 f7 d8 64 89 01 48 RSP: 002b:00007ffc1c177248 EFLAGS: 00000206 ORIG_RAX: 0000000000000032 RAX: ffffffffffffffda RBX: 0000000020001550 RCX: 00007f8ac051de5d RDX: ffffffffffffff80 RSI: 0000000000000000 RDI: 0000000000000004 RBP: 00007ffc1c177270 R08: 0000000000000018 R09: 0000000000000007 R10: 0000000020001540 R11: 0000000000000206 R12: 00007ffc1c177388 R13: 0000000000401169 R14: 0000000000403e18 R15: 00007f8ac0723000 </TASK>
--- include/net/inet_hashtables.h | 1 + net/dccp/ipv4.c | 3 +-- net/dccp/ipv6.c | 3 +-- net/dccp/proto.c | 3 +-- net/ipv4/inet_hashtables.c | 35 ++++++++++++++++++++++++++++++++--- net/ipv4/tcp.c | 3 +-- net/ipv4/tcp_ipv4.c | 3 +-- net/ipv6/tcp_ipv6.c | 3 +-- 8 files changed, 39 insertions(+), 15 deletions(-)
include/net/inet_hashtables.h +1 -0
@@ -282,6 +282,7 @@ inet_bhash2_addr_any_hashbucket(const struct sock *sk, const struct net *net, in* rcv_saddr field should already have been updated when this is called. */ int inet_bhash2_update_saddr (struct sock *sk,void *saddr,int family); +void inet_bhash2_reset_saddr (struct sock *sk);void inet_bind_hash (struct sock *sk,struct inet_bind_bucket *tb,struct inet_bind2_bucket *tb2,unsigned short port);
net/dccp/ipv4.c +1 -2
@@ -143,8 +143,7 @@ int dccp_v4_connect(struct sock *sk, struct sockaddr *uaddr, int addr_len)* This unhashes the socket and releases the local port, if necessary. */ dccp_set_state (sk,DCCP_CLOSED ); -if (!(sk->sk_userlocks &SOCK_BINDADDR_LOCK )) -inet_reset_saddr (sk); +inet_bhash2_reset_saddr (sk);ip_rt_put (rt); sk->sk_route_caps =0 ; inet->inet_dport =0 ;
net/dccp/ipv6.c +1 -2
@@ -970,8 +970,7 @@ static int dccp_v6_connect(struct sock *sk, struct sockaddr *uaddr, late_failure:dccp_set_state (sk,DCCP_CLOSED ); -if (!(sk->sk_userlocks &SOCK_BINDADDR_LOCK )) -inet_reset_saddr (sk); +inet_bhash2_reset_saddr (sk);__sk_dst_reset (sk); failure: inet->inet_dport =0 ;
net/dccp/proto.c +1 -2
@@ -279,8 +279,7 @@ int dccp_disconnect(struct sock *sk, int flags) inet->inet_dport =0 ; -if (!(sk->sk_userlocks &SOCK_BINDADDR_LOCK )) -inet_reset_saddr (sk); +inet_bhash2_reset_saddr (sk); sk->sk_shutdown =0 ;sock_reset_flag (sk,SOCK_DONE );
net/ipv4/inet_hashtables.c +32 -3
@@ -871,7 +871,7 @@ static void inet_update_saddr(struct sock *sk, void *saddr, int family)#endif } -int inet_bhash2_update_saddr (struct sock *sk,void *saddr,int family) +static int __inet_bhash2_update_saddr (struct sock *sk,void *saddr,int family,bool reset) {struct inet_hashinfo *hinfo =tcp_or_dccp_get_hashinfo (sk);struct inet_bind_hashbucket *head, *head2;
@@ -883,7 +883,11 @@ int inet_bhash2_update_saddr(struct sock *sk, void *saddr, int family)if (!inet_csk (sk)->icsk_bind2_hash) {/* Not bind()ed before. */ -inet_update_saddr (sk, saddr, family); +if (reset) +inet_reset_saddr (sk); +else +inet_update_saddr (sk, saddr, family); + goto out; }
@@ -901,6 +905,16 @@ int inet_bhash2_update_saddr(struct sock *sk, void *saddr, int family)*/ new_tb2 =kmem_cache_alloc (hinfo->bind2_bucket_cachep,GFP_ATOMIC );if (!new_tb2) { +if (reset) { +/* The (INADDR_ANY, port) bucket might have already +* been freed, then we cannot fixup icsk_bind2_hash, +* so we give up and unlink sk from bhash/bhash2 not +* to leave inconsistency in bhash2. +*/ +inet_put_port (sk); +inet_reset_saddr (sk); + } + err = -ENOMEM ; goto unlock; }
@@ -912,7 +926,10 @@ int inet_bhash2_update_saddr(struct sock *sk, void *saddr, int family)inet_bind2_bucket_destroy (hinfo->bind2_bucket_cachep,inet_csk (sk)->icsk_bind2_hash);spin_unlock (&head2->lock); -inet_update_saddr (sk, saddr, family); +if (reset) +inet_reset_saddr (sk); +else +inet_update_saddr (sk, saddr, family); head2 =inet_bhashfn_portaddr (hinfo, sk, net, port);
@@ -934,8 +951,20 @@ int inet_bhash2_update_saddr(struct sock *sk, void *saddr, int family) out:return err; } + +int inet_bhash2_update_saddr (struct sock *sk,void *saddr,int family) +{ +return __inet_bhash2_update_saddr (sk, saddr, family, false); +}EXPORT_SYMBOL_GPL (inet_bhash2_update_saddr); +void inet_bhash2_reset_saddr (struct sock *sk) +{ +if (!(sk->sk_userlocks &SOCK_BINDADDR_LOCK )) +__inet_bhash2_update_saddr (sk,NULL ,0 , true); +} +EXPORT_SYMBOL_GPL (inet_bhash2_reset_saddr); +/* RFC 6056 3.3.4. Algorithm 4: Double-Hash Port Selection Algorithm * Note that we use 32bit integers (vs RFC 'short integers') * because 2^16 is not a multiple of num_ephemeral and this
net/ipv4/tcp.c +1 -2
@@ -3114,8 +3114,7 @@ int tcp_disconnect(struct sock *sk, int flags) inet->inet_dport =0 ; -if (!(sk->sk_userlocks &SOCK_BINDADDR_LOCK )) -inet_reset_saddr (sk); +inet_bhash2_reset_saddr (sk); sk->sk_shutdown =0 ;sock_reset_flag (sk,SOCK_DONE );
net/ipv4/tcp_ipv4.c +1 -2
@@ -331,8 +331,7 @@ int tcp_v4_connect(struct sock *sk, struct sockaddr *uaddr, int addr_len)* if necessary. */ tcp_set_state (sk,TCP_CLOSE ); -if (!(sk->sk_userlocks &SOCK_BINDADDR_LOCK )) -inet_reset_saddr (sk); +inet_bhash2_reset_saddr (sk);ip_rt_put (rt); sk->sk_route_caps =0 ; inet->inet_dport =0 ;
net/ipv6/tcp_ipv6.c +1 -2
@@ -346,8 +346,7 @@ static int tcp_v6_connect(struct sock *sk, struct sockaddr *uaddr, late_failure:tcp_set_state (sk,TCP_CLOSE ); -if (!(sk->sk_userlocks &SOCK_BINDADDR_LOCK )) -inet_reset_saddr (sk); +inet_bhash2_reset_saddr (sk); failure: inet->inet_dport =0 ; sk->sk_route_caps =0 ;
Signature · 1 line
2.30.2