udp_tunnel: Speed up UDP tunnel device destruction (Part I)
Subsystems (12)
cover udp_tunnel: Speed up UDP tunnel device destruction (Part I)
Most of the UDP tunnel devices call synchronize_rcu() twice during destruction, for example, vxlan has
1) synchronize_rcu() in udp_tunnel_sock_release()
2) synchronize_net() in vxlan_sock_release()
The goal of this series is to remove the former, and another followup series removes the latter.
synchronize_rcu() was added in udp_tunnel_sock_release() by commit 3cf7203ca620 ("net/tunnel: wait until all sk_user_data reader finish before releasing the sock").
This was intended to protect the fast path of a dying vxlan from dereferencing vxlan_sock->sock->sk after sock_orphan() has set sock->sk to NULL.
Most of the UDP tunnel devices store struct socket to its private struct, but it is NOT needed in the fast paths; struct sock is used there, but struct socket is only used for tunnel setup / teardown.
This is probably because UDP tunnel functions accept struct socket, but even such functions do not need it, except for udp_tunnel_sock_release(), which can safely access sk->sk_socket.
The overview of the series:
Patch 1 - 5 : Convert UDP tunnel helper to take struct sock
Patch 6 : Small fix for 10-years-old bug
Patch 7 - 14 : Store struct sock in tunnel devices
Patch 15 : Remove synchronize_rcu() in udp_tunnel_sock_release()
(I noted bugs of fou and amt, so Sashiko may point out
such pre-existing issues in fou, amt, .. and pfcp and tipc, but they are orthogonal and I can follow up separately.)
With this change, a script creating/upping vxlan in 4000 netns runs 10x faster.
$ cat vxlan.sh
for i in `seq 1 40`
do
(for j in `seq 1 100` ; do
unshare -n bash -c "ip link add vxlan0 type vxlan id 100 local 127.0.0.1 dstport 4789 && ip link set vxlan0 up";
done) &
done
waitWith bpftrace, we can see vxlan_stop() is significantly faster too.
bpftrace -e '
kprobe:vxlan_stop {
@start[tid] = nsecs;
}
kretprobe:vxlan_stop /@start[tid]/ {
@duration_us = hist((nsecs - @start[tid]) / 1000);
delete(@start[tid]);
}
END {
printf("\nExecution time of vxlan_stop (us):\n");
}'Before:
# time ./vxlan.sh // without bpftrace real 0m50.615s user 0m8.171s sys 1m45.101s
@duration_us:
[4K, 8K) 1266 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ | [8K, 16K) 1957 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@| [16K, 32K) 764 |@@@@@@@@@@@@@@@@@@@@ | [32K, 64K) 6 | | [64K, 128K) 4 | | [128K, 256K) 3 | |
After:
# time ./vxlan.sh // without bpftrace real 0m5.247s user 0m7.956s sys 1m47.404s
@duration_us:
[16, 32) 3411 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@| [32, 64) 383 |@@@@@ | [64, 128) 107 |@ | [128, 256) 79 |@ | [256, 512) 16 | | [512, 1K) 2 | | [1K, 2K) 2 | |
Kuniyuki Iwashima (15):
udp_tunnel: Pass struct sock to udp_tunnel_sock_release().
udp_tunnel: Pass struct sock to setup_udp_tunnel_sock().
udp_tunnel: Pass struct sock to udp_tunnel6_dst_lookup().
udp_tunnel: Pass struct sock to udp_tunnel_{push,drop}_rx_port().
udp_tunnel: Pass struct sock to udp_tunnel_notify_{add,del}_rx_port().
vxlan: Fix potential null-ptr-deref in vxlan_gro_prepare_receive().
vxlan: Store struct sock in struct vxlan_sock.
vxlan: Free vxlan_sock with kfree_rcu().
geneve: Store struct sock in struct geneve_sock.
bareudp: Store struct sock in struct bareudp_dev.
fou: Store struct sock in struct fou.
amt: Store struct sock in struct amt_dev.
pfcp: Store struct sock in struct pfcp_dev.
tipc: Store struct sock in struct udp_bearer.
udp_tunnel: Remove synchronize_rcu() in udp_tunnel_sock_release().
drivers/infiniband/sw/rxe/rxe_net.c | 6 +-- drivers/infiniband/sw/rxe/rxe_ns.c | 4 +- drivers/net/amt.c | 80 ++++++++++++++--------------- drivers/net/bareudp.c | 51 +++++++++--------- drivers/net/geneve.c | 54 +++++++++---------- drivers/net/gtp.c | 10 ++-- drivers/net/ovpn/udp.c | 2 +- drivers/net/pfcp.c | 17 +++--- drivers/net/vxlan/vxlan_core.c | 60 ++++++++++++---------- drivers/net/vxlan/vxlan_multicast.c | 8 +-- drivers/net/wireguard/socket.c | 8 +-- include/net/amt.h | 2 +- include/net/udp_tunnel.h | 14 ++--- include/net/vxlan.h | 5 +- net/ipv4/fou_core.c | 21 ++++---- net/ipv4/udp_tunnel_core.c | 23 ++++----- net/ipv6/ip6_udp_tunnel.c | 6 +-- net/l2tp/l2tp_core.c | 2 +- net/rxrpc/local_object.c | 2 +- net/sctp/protocol.c | 10 ++-- net/tipc/udp_media.c | 32 +++++++----- 21 files changed, 209 insertions(+), 208 deletions(-)
Signature · 1 line
2.54.0.545.g6539524ca2-goog
PATCH 1/15 udp_tunnel: Pass struct sock to udp_tunnel_sock_release().
None of the udp_tunnel users need struct socket in their fast paths; it is only used for tunnel setup / teardown.
While the UDP tunnel interface accepts struct socket, this encourages users to store the pointer unnecessarily. This leads to extra dereferences when accessing struct sock fields (e.g., sk->sk_user_data instead of sock->sk->sk_user_data).
Furthermore, these dereferences necessitate synchronize_rcu() in udp_tunnel_sock_release() to protect the fast paths from sock_orphan() setting sk->sk_socket to NULL.
This overhead can be avoided if users store the struct sock pointer directly in their private structures.
As a prep, let's change udp_tunnel_sock_release() to take struct sock instead of struct socket.
--- drivers/infiniband/sw/rxe/rxe_net.c | 4 ++-- drivers/infiniband/sw/rxe/rxe_ns.c | 4 ++-- drivers/net/amt.c | 2 +- drivers/net/bareudp.c | 2 +- drivers/net/geneve.c | 2 +- drivers/net/gtp.c | 6 +++--- drivers/net/pfcp.c | 2 +- drivers/net/vxlan/vxlan_core.c | 4 ++-- drivers/net/wireguard/socket.c | 4 ++-- include/net/udp_tunnel.h | 2 +- net/ipv4/fou_core.c | 7 ++----- net/ipv4/udp_tunnel_core.c | 6 ++++-- net/sctp/protocol.c | 6 +++--- net/tipc/udp_media.c | 4 ++-- 14 files changed, 27 insertions(+), 28 deletions(-)
drivers/infiniband/sw/rxe/rxe_net.c +2 -2
@@ -288,7 +288,7 @@ static struct socket *rxe_setup_udp_tunnel(struct net *net, __be16 port,return sock; } -static void rxe_release_udp_tunnel (struct socket *sk) +static void rxe_release_udp_tunnel (struct sock *sk) {if (sk)udp_tunnel_sock_release (sk);
@@ -636,7 +636,7 @@ static void rxe_sock_put(struct sock *sk,if (refcount_read (&sk->sk_refcnt) >SK_REF_FOR_TUNNEL ) {__sock_put (sk); }else { -rxe_release_udp_tunnel (sk->sk_socket); +rxe_release_udp_tunnel (sk); sk =NULL ;set_sk (net, sk); }
drivers/infiniband/sw/rxe/rxe_ns.c +2 -2
@@ -47,7 +47,7 @@ static void rxe_ns_exit(struct net *net)rcu_read_unlock ();if (sk) {rcu_assign_pointer (ns_sk->rxe_sk4,NULL ); -udp_tunnel_sock_release (sk->sk_socket); +udp_tunnel_sock_release (sk); }#if IS_ENABLED (CONFIG_IPV6 )
@@ -56,7 +56,7 @@ static void rxe_ns_exit(struct net *net)rcu_read_unlock ();if (sk) {rcu_assign_pointer (ns_sk->rxe_sk6,NULL ); -udp_tunnel_sock_release (sk->sk_socket); +udp_tunnel_sock_release (sk); }#endif }
drivers/net/amt.c +1 -1
@@ -3032,7 +3032,7 @@ static int amt_dev_stop(struct net_device *dev)RCU_INIT_POINTER (amt->sock,NULL );synchronize_net ();if (sock) -udp_tunnel_sock_release (sock); +udp_tunnel_sock_release (sock->sk);cancel_work_sync (&amt->event_wq);for (i =0 ; i <AMT_MAX_EVENTS ; i++) {
drivers/net/bareudp.c +1 -1
@@ -290,7 +290,7 @@ static void bareudp_sock_release(struct bareudp_dev *bareudp) sock = bareudp->sock;rcu_assign_pointer (bareudp->sock,NULL );synchronize_net (); -udp_tunnel_sock_release (sock); +udp_tunnel_sock_release (sock->sk); }static int bareudp_stop (struct net_device *dev)
drivers/net/geneve.c +1 -1
@@ -1018,7 +1018,7 @@ static void __geneve_sock_release(struct geneve_sock *gs)list_del (&gs->list);udp_tunnel_notify_del_rx_port (gs->sock,UDP_TUNNEL_TYPE_GENEVE ); -udp_tunnel_sock_release (gs->sock); +udp_tunnel_sock_release (gs->sock->sk);kfree_rcu (gs, rcu); }
drivers/net/gtp.c +3 -3
@@ -885,8 +885,8 @@ static void gtp_encap_disable_sock(struct sock *sk)static void gtp_encap_disable (struct gtp_dev *gtp) {if (gtp->sk_created) { -udp_tunnel_sock_release (gtp->sk0->sk_socket); -udp_tunnel_sock_release (gtp->sk1u->sk_socket); +udp_tunnel_sock_release (gtp->sk0); +udp_tunnel_sock_release (gtp->sk1u); gtp->sk_created = false; gtp->sk0 =NULL ; gtp->sk1u =NULL ;
@@ -1451,7 +1451,7 @@ static int gtp_create_sockets(struct gtp_dev *gtp, const struct nlattr *nla, sk1u =gtp_create_sock (UDP_ENCAP_GTP1U , gtp, nla, family);if (IS_ERR (sk1u)) { -udp_tunnel_sock_release (sk0->sk_socket); +udp_tunnel_sock_release (sk0);return PTR_ERR (sk1u); }
drivers/net/pfcp.c +1 -1
@@ -104,7 +104,7 @@ static int pfcp_encap_recv(struct sock *sk, struct sk_buff *skb)static void pfcp_del_sock (struct pfcp_dev *pfcp) { -udp_tunnel_sock_release (pfcp->sock); +udp_tunnel_sock_release (pfcp->sock->sk); pfcp->sock =NULL ; }
drivers/net/vxlan/vxlan_core.c +2 -2
@@ -1519,13 +1519,13 @@ static void vxlan_sock_release(struct vxlan_dev *vxlan)vxlan_vs_del_dev (vxlan);if (__vxlan_sock_release_prep (sock4)) { -udp_tunnel_sock_release (sock4->sock); +udp_tunnel_sock_release (sock4->sock->sk);kfree (sock4); }#if IS_ENABLED (CONFIG_IPV6 )if (__vxlan_sock_release_prep (sock6)) { -udp_tunnel_sock_release (sock6->sock); +udp_tunnel_sock_release (sock6->sock->sk);kfree (sock6); }#endif
drivers/net/wireguard/socket.c +2 -2
@@ -335,7 +335,7 @@ static void sock_free(struct sock *sock)if (unlikely (!sock))return ;sk_clear_memalloc (sock); -udp_tunnel_sock_release (sock->sk_socket); +udp_tunnel_sock_release (sock); }static void set_sock_opts (struct socket *sock)
@@ -396,7 +396,7 @@ int wg_socket_init(struct wg_device *wg, u16 port) port6.local_udp_port =inet_sk (new4->sk)->inet_sport; ret =udp_sock_create (net, &port6, &new6);if (ret <0 ) { -udp_tunnel_sock_release (new4); +udp_tunnel_sock_release (new4->sk);if (ret == -EADDRINUSE && !port && retries++ <100 ) goto retry; pr_err("%s: Could not create IPv6 socket\n",
include/net/udp_tunnel.h +1 -1
@@ -176,7 +176,7 @@ static inline void udp_tunnel_set_inner_protocol(struct sk_buff *skb,skb_set_inner_protocol (skb, inner_proto); } -void udp_tunnel_sock_release (struct socket *sock); +void udp_tunnel_sock_release (struct sock *sk); struct rtable *udp_tunnel_dst_lookup(struct sk_buff *skb, struct net_device *dev,
net/ipv4/fou_core.c +2 -5
@@ -558,11 +558,8 @@ static int fou_add_to_port_list(struct net *net, struct fou *fou,static void fou_release (struct fou *fou) { -struct socket *sock = fou->sock; -list_del (&fou->list); -udp_tunnel_sock_release (sock); - +udp_tunnel_sock_release (fou->sock->sk);kfree_rcu (fou, rcu); }
@@ -634,7 +631,7 @@ static int fou_create(struct net *net, struct fou_cfg *cfg, error:kfree (fou);if (sock) -udp_tunnel_sock_release (sock); +udp_tunnel_sock_release (sock->sk);return err; }
net/ipv4/udp_tunnel_core.c +4 -2
@@ -195,9 +195,11 @@ void udp_tunnel_xmit_skb(struct rtable *rt, struct sock *sk, struct sk_buff *skb }EXPORT_SYMBOL_GPL (udp_tunnel_xmit_skb); -void udp_tunnel_sock_release (struct socket *sock) +void udp_tunnel_sock_release (struct sock *sk) { -rcu_assign_sk_user_data (sock->sk,NULL ); +struct socket *sock = sk->sk_socket; + +rcu_assign_sk_user_data (sk,NULL );synchronize_rcu ();kernel_sock_shutdown (sock,SHUT_RDWR );sock_release (sock);
net/sctp/protocol.c +3 -3
@@ -878,7 +878,7 @@ int sctp_udp_sock_start(struct net *net) err =udp_sock_create (net, &udp_conf, &sock);if (err) {pr_err ("Failed to create the SCTP UDP tunneling v6 sock\n" ); -udp_tunnel_sock_release (net->sctp.udp4_sock->sk_socket); +udp_tunnel_sock_release (net->sctp.udp4_sock); net->sctp.udp4_sock =NULL ;return err; }
@@ -896,11 +896,11 @@ int sctp_udp_sock_start(struct net *net)void sctp_udp_sock_stop (struct net *net) {if (net->sctp.udp4_sock) { -udp_tunnel_sock_release (net->sctp.udp4_sock->sk_socket); +udp_tunnel_sock_release (net->sctp.udp4_sock); net->sctp.udp4_sock =NULL ; }if (net->sctp.udp6_sock) { -udp_tunnel_sock_release (net->sctp.udp6_sock->sk_socket); +udp_tunnel_sock_release (net->sctp.udp6_sock); net->sctp.udp6_sock =NULL ; } }
net/tipc/udp_media.c +2 -2
@@ -793,7 +793,7 @@ static int tipc_udp_enable(struct net *net, struct tipc_bearer *b, free:dst_cache_destroy (&ub->rcast.dst_cache); -udp_tunnel_sock_release (ub->ubsock); +udp_tunnel_sock_release (ub->ubsock->sk); err:kfree (ub);return err;
@@ -815,7 +815,7 @@ static void cleanup_bearer(struct work_struct *work) tn =tipc_net (sock_net (ub->ubsock->sk));dst_cache_destroy (&ub->rcast.dst_cache); -udp_tunnel_sock_release (ub->ubsock); +udp_tunnel_sock_release (ub->ubsock->sk);/* Note: could use a call_rcu() to avoid another synchronize_net() */ synchronize_net ();
Signature · 1 line
2.54.0.545.g6539524ca2-goog
Re: [PATCH v1 net-next 01/15] udp_tunnel: Pass struct sock to udp_tunnel_sock_release().
For Sashiko's review:
On Fri, May 1, 2026 at 8:14 PM Kuniyuki Iwashima <[email protected]> wrote:
[...]
Quoted 14 lines
drivers/infiniband/sw/rxe/rxe_net.c +1
-1@@ -288,7 +288,7 @@ static struct socket *rxe_setup_udp_tunnel(struct net *net, __be16 port,return sock; } -static void rxe_release_udp_tunnel (struct socket *sk) +static void rxe_release_udp_tunnel (struct sock *sk) {if (sk)udp_tunnel_sock_release (sk);
drivers/infiniband/sw/rxe/rxe_net.c static void rxe_sock_put(struct sock *sk, · line 642
Quoted 8 lines
drivers/infiniband/sw/rxe/rxe_net.c +1
-1@@ -636,7 +636,7 @@ static void rxe_sock_put(struct sock *sk,if (refcount_read (&sk->sk_refcnt) >SK_REF_FOR_TUNNEL ) {__sock_put (sk); }else { -rxe_release_udp_tunnel (sk->sk_socket); +rxe_release_udp_tunnel (sk); sk =NULL ;set_sk (net, sk); }
---8<---
This isn't a bug introduced by this patch, but does this assume sk_refcnt exclusively tracks rxe endpoints?
If the network stack holds a transient reference via sock_hold() when this is called, it appears this would decrement the refcount and skip releasing the tunnel.
Could this lead to a permanent socket leak when the network stack drops its reference?
---8<---
Pre-existing issue.
The socket management in RXE is fundamentally broken (vulnerable to concurrent newlink/dellink), and I'm working on a fix here.
https://lore.kernel.org/all/[email protected]/
drivers/infiniband/sw/rxe/rxe_ns.c static void rxe_ns_exit(struct net *net) · line 52
Quoted 12 lines
drivers/infiniband/sw/rxe/rxe_ns.c +1
-1@@ -47,7 +47,7 @@ static void rxe_ns_exit(struct net *net)rcu_read_unlock ();if (sk) {rcu_assign_pointer (ns_sk->rxe_sk4,NULL ); -udp_tunnel_sock_release (sk->sk_socket); +udp_tunnel_sock_release (sk); }
---8<---
This isn't a bug introduced by this patch, but could dereferencing sk here cause a use-after-free?
The sk pointer is obtained via rcu_dereference() but used after rcu_read_unlock() completes:rcu_read_lock();
sk = rcu_dereference(ns_sk->rxe_sk4);rcu_read_unlock();
If a concurrent thread drops the last reference to the socket right after the lock is released, could sk point to freed memory during the udp_tunnel_sock_release() call?
---8<---
Same RXE pre-existing issue.
[...]
Quoted 4 lines
net/ipv4/fou_core.c +0
-0
[...]
net/ipv4/fou_core.c static int fou_create(struct net *net, struct fou_cfg *cfg, · line 637
Quoted 9 lines
net/ipv4/fou_core.c +1
-1@@ -634,7 +631,7 @@ static int fou_create(struct net *net, struct fou_cfg *cfg, error:kfree (fou);if (sock) -udp_tunnel_sock_release (sock); +udp_tunnel_sock_release (sock->sk);return err; }
---8<---
This isn't a bug introduced by this patch, but is there a use-after-free risk in this error path?
Because kfree(fou) is called before udp_tunnel_sock_release(), if a UDP packet is received concurrently, could the network softirq invoke fou_udp_recv() and access the freed fou object via rcu_dereference_sk_user_data(sk)? Should kfree(fou) be called after the release, or replaced with kfree_rcu()? ---8<---
Pre-existing issue that I mentioned in patch 11 and orthogonal to this series.
Quoted 8 lines
net/ipv4/udp_tunnel_core.c +0
-0@@ -195,9 +195,11 @@ void udp_tunnel_xmit_skb(struct rtable *rt, struct sock *sk, struct sk_buff *skb }EXPORT_SYMBOL_GPL (udp_tunnel_xmit_skb);
net/ipv4/udp_tunnel_core.c void udp_tunnel_xmit_skb(struct rtable *rt, struct sock *sk, struct sk_buff *skb · line 205
Quoted 10 lines
net/ipv4/udp_tunnel_core.c +4
-2-void udp_tunnel_sock_release (struct socket *sock) +void udp_tunnel_sock_release (struct sock *sk) { -rcu_assign_sk_user_data (sock->sk,NULL ); +struct socket *sock = sk->sk_socket; + +rcu_assign_sk_user_data (sk,NULL );synchronize_rcu ();kernel_sock_shutdown (sock,SHUT_RDWR );sock_release (sock);
---8<---
Can kernel_sock_shutdown() cause a NULL pointer dereference here? ---8<---
No, this function is not supposed to be called more than once. Sashiko's example is due to broken RXE.
PATCH 2/15 udp_tunnel: Pass struct sock to setup_udp_tunnel_sock().
In reply to Kuniyuki IwashimaNone of the udp_tunnel users need struct socket in their fast paths; it is only used for tunnel setup / teardown.
Even setup_udp_tunnel_sock() does not need struct socket.
Let's change setup_udp_tunnel_sock() to take struct sock instead of struct socket.
--- drivers/infiniband/sw/rxe/rxe_net.c | 2 +- drivers/net/amt.c | 2 +- drivers/net/bareudp.c | 2 +- drivers/net/geneve.c | 2 +- drivers/net/gtp.c | 4 ++-- drivers/net/ovpn/udp.c | 2 +- drivers/net/pfcp.c | 2 +- drivers/net/vxlan/vxlan_core.c | 2 +- drivers/net/wireguard/socket.c | 4 ++-- include/net/udp_tunnel.h | 2 +- net/ipv4/fou_core.c | 2 +- net/ipv4/udp_tunnel_core.c | 4 +--- net/l2tp/l2tp_core.c | 2 +- net/rxrpc/local_object.c | 2 +- net/sctp/protocol.c | 4 ++-- net/tipc/udp_media.c | 2 +- 16 files changed, 19 insertions(+), 21 deletions(-)
drivers/infiniband/sw/rxe/rxe_net.c +1 -1
@@ -283,7 +283,7 @@ static struct socket *rxe_setup_udp_tunnel(struct net *net, __be16 port, tnl_cfg.encap_rcv = rxe_udp_encap_recv;/* Setup UDP tunnel */ -setup_udp_tunnel_sock (net, sock, &tnl_cfg); +setup_udp_tunnel_sock (net, sock->sk, &tnl_cfg);return sock; }
drivers/net/amt.c +1 -1
@@ -2979,7 +2979,7 @@ static int amt_socket_create(struct amt_dev *amt) tunnel_cfg.encap_rcv = amt_rcv; tunnel_cfg.encap_err_lookup = amt_err_lookup; tunnel_cfg.encap_destroy =NULL ; -setup_udp_tunnel_sock (amt->net, sock, &tunnel_cfg); +setup_udp_tunnel_sock (amt->net, sock->sk, &tunnel_cfg);rcu_assign_pointer (amt->sock, sock);return 0 ;
drivers/net/bareudp.c +1 -1
@@ -268,7 +268,7 @@ static int bareudp_socket_create(struct bareudp_dev *bareudp, __be16 port) tunnel_cfg.encap_rcv = bareudp_udp_encap_recv; tunnel_cfg.encap_err_lookup = bareudp_err_lookup; tunnel_cfg.encap_destroy =NULL ; -setup_udp_tunnel_sock (bareudp->net, sock, &tunnel_cfg); +setup_udp_tunnel_sock (bareudp->net, sock->sk, &tunnel_cfg);rcu_assign_pointer (bareudp->sock, sock);return 0 ;
drivers/net/geneve.c +1 -1
@@ -1006,7 +1006,7 @@ static struct geneve_sock *geneve_socket_create(struct net *net, __be16 port, tunnel_cfg.encap_rcv = geneve_udp_encap_recv; tunnel_cfg.encap_err_lookup = geneve_udp_encap_err_lookup; tunnel_cfg.encap_destroy =NULL ; -setup_udp_tunnel_sock (net, sock, &tunnel_cfg); +setup_udp_tunnel_sock (net, sock->sk, &tunnel_cfg);list_add (&gs->list, &gn->sock_list);return gs; }
drivers/net/gtp.c +2 -2
@@ -1434,7 +1434,7 @@ static struct sock *gtp_create_sock(int type, struct gtp_dev *gtp, tuncfg.encap_rcv = gtp_encap_recv; tuncfg.encap_destroy =NULL ; -setup_udp_tunnel_sock (net, sock, &tuncfg); +setup_udp_tunnel_sock (net, sock->sk, &tuncfg);return sock->sk; }
@@ -1689,7 +1689,7 @@ static struct sock *gtp_encap_enable_socket(int fd, int type, tuncfg.encap_rcv = gtp_encap_recv; tuncfg.encap_destroy = gtp_encap_destroy; -setup_udp_tunnel_sock (sock_net (sock->sk), sock, &tuncfg); +setup_udp_tunnel_sock (sock_net (sock->sk), sk, &tuncfg); out_rel_sock:release_sock (sock->sk);
drivers/net/ovpn/udp.c +1 -1
@@ -399,7 +399,7 @@ int ovpn_udp_socket_attach(struct ovpn_socket *ovpn_sock, struct socket *sock,if (!old_data) {/* socket is currently unused - we can take it */ rcu_read_unlock (); -setup_udp_tunnel_sock (sock_net (ovpn_sock->sk), sock, &cfg); +setup_udp_tunnel_sock (sock_net (ovpn_sock->sk), sock->sk, &cfg);return 0 ; }
drivers/net/pfcp.c +1 -1
@@ -172,7 +172,7 @@ static struct socket *pfcp_create_sock(struct pfcp_dev *pfcp) tuncfg.encap_rcv = pfcp_encap_recv; tuncfg.encap_type =1 ; -setup_udp_tunnel_sock (net, sock, &tuncfg); +setup_udp_tunnel_sock (net, sock->sk, &tuncfg);return sock; }
drivers/net/vxlan/vxlan_core.c +1 -1
@@ -3620,7 +3620,7 @@ static struct vxlan_sock *vxlan_socket_create(struct net *net, bool ipv6, tunnel_cfg.gro_complete = vxlan_gro_complete; } -setup_udp_tunnel_sock (net, sock, &tunnel_cfg); +setup_udp_tunnel_sock (net, sock->sk, &tunnel_cfg);return vs; }
drivers/net/wireguard/socket.c +2 -2
@@ -389,7 +389,7 @@ int wg_socket_init(struct wg_device *wg, u16 port) goto out; }set_sock_opts (new4); -setup_udp_tunnel_sock (net, new4, &cfg); +setup_udp_tunnel_sock (net, new4->sk, &cfg);#if IS_ENABLED (CONFIG_IPV6 )if (ipv6_mod_enabled ()) {
@@ -404,7 +404,7 @@ int wg_socket_init(struct wg_device *wg, u16 port) goto out; }set_sock_opts (new6); -setup_udp_tunnel_sock (net, new6, &cfg); +setup_udp_tunnel_sock (net, new6->sk, &cfg); }#endif
include/net/udp_tunnel.h +1 -1
@@ -94,7 +94,7 @@ struct udp_tunnel_sock_cfg { };/* Setup the given (UDP) sock to receive UDP encapsulated packets */ -void setup_udp_tunnel_sock (struct net *net,struct socket *sock, +void setup_udp_tunnel_sock (struct net *net,struct sock *sk,struct udp_tunnel_sock_cfg *sock_cfg);/* -- List of parsable UDP tunnel types --
net/ipv4/fou_core.c +1 -1
@@ -615,7 +615,7 @@ static int fou_create(struct net *net, struct fou_cfg *cfg, goto error; } -setup_udp_tunnel_sock (net, sock, &tunnel_cfg); +setup_udp_tunnel_sock (net, sk, &tunnel_cfg); sk->sk_allocation =GFP_ATOMIC ;
net/ipv4/udp_tunnel_core.c +1 -3
@@ -68,11 +68,9 @@ static bool sk_saddr_any(struct sock *sk)#endif } -void setup_udp_tunnel_sock (struct net *net,struct socket *sock, +void setup_udp_tunnel_sock (struct net *net,struct sock *sk,struct udp_tunnel_sock_cfg *cfg) { -struct sock *sk = sock->sk; -/* Disable multicast loopback */ inet_clear_bit (MC_LOOP , sk);
net/l2tp/l2tp_core.c +1 -1
@@ -1683,7 +1683,7 @@ int l2tp_tunnel_register(struct l2tp_tunnel *tunnel, struct net *net, .encap_destroy = l2tp_udp_encap_destroy, }; -setup_udp_tunnel_sock (net, sock, &udp_cfg); +setup_udp_tunnel_sock (net, sock->sk, &udp_cfg); } sk->sk_allocation =GFP_ATOMIC ;
net/rxrpc/local_object.c +1 -1
@@ -194,7 +194,7 @@ static int rxrpc_open_socket(struct rxrpc_local *local, struct net *net) tuncfg.encap_rcv = rxrpc_encap_rcv; tuncfg.encap_err_rcv = rxrpc_encap_err_rcv; tuncfg.sk_user_data = local; -setup_udp_tunnel_sock (net, local->socket, &tuncfg); +setup_udp_tunnel_sock (net, local->socket->sk, &tuncfg);/* set the socket up */ usk = local->socket->sk;
net/sctp/protocol.c +2 -2
@@ -864,7 +864,7 @@ int sctp_udp_sock_start(struct net *net) tuncfg.encap_type =1 ; tuncfg.encap_rcv = sctp_udp_rcv; tuncfg.encap_err_lookup = sctp_udp_v4_err; -setup_udp_tunnel_sock (net, sock, &tuncfg); +setup_udp_tunnel_sock (net, sock->sk, &tuncfg); net->sctp.udp4_sock = sock->sk;#if IS_ENABLED (CONFIG_IPV6 )
@@ -886,7 +886,7 @@ int sctp_udp_sock_start(struct net *net) tuncfg.encap_type =1 ; tuncfg.encap_rcv = sctp_udp_rcv; tuncfg.encap_err_lookup = sctp_udp_v6_err; -setup_udp_tunnel_sock (net, sock, &tuncfg); +setup_udp_tunnel_sock (net, sock->sk, &tuncfg); net->sctp.udp6_sock = sock->sk;#endif
net/tipc/udp_media.c +1 -1
@@ -771,7 +771,7 @@ static int tipc_udp_enable(struct net *net, struct tipc_bearer *b, tuncfg.encap_type =1 ; tuncfg.encap_rcv = tipc_udp_recv; tuncfg.encap_destroy =NULL ; -setup_udp_tunnel_sock (net, ub->ubsock, &tuncfg); +setup_udp_tunnel_sock (net, ub->ubsock->sk, &tuncfg); err =dst_cache_init (&ub->rcast.dst_cache,GFP_ATOMIC );if (err)
Signature · 1 line
2.54.0.545.g6539524ca2-goog
PATCH 3/15 udp_tunnel: Pass struct sock to udp_tunnel6_dst_lookup().
In reply to Kuniyuki IwashimaNone of the udp_tunnel users need struct socket in their fast paths; it is only used for tunnel setup / teardown.
Even udp_tunnel6_dst_lookup() does not need struct socket.
Let's change udp_tunnel6_dst_lookup() to take struct sock instead of struct socket.
--- drivers/net/bareudp.c | 4 ++-- drivers/net/geneve.c | 4 ++-- drivers/net/vxlan/vxlan_core.c | 4 ++-- include/net/udp_tunnel.h | 2 +- net/ipv6/ip6_udp_tunnel.c | 6 +++--- 5 files changed, 10 insertions(+), 10 deletions(-)
drivers/net/bareudp.c +2 -2
@@ -396,7 +396,7 @@ static int bareudp6_xmit_skb(struct sk_buff *skb, struct net_device *dev, sport =udp_flow_src_port (bareudp->net, skb, bareudp->sport_min,USHRT_MAX , true); - dst =udp_tunnel6_dst_lookup (skb, dev, bareudp->net, sock,0 , &saddr, + dst =udp_tunnel6_dst_lookup (skb, dev, bareudp->net, sock->sk,0 , &saddr, key, sport, bareudp->port, key->tos, use_cache ? (struct dst_cache *) &info->dst_cache :NULL );
@@ -532,7 +532,7 @@ static int bareudp_fill_metadata_dst(struct net_device *dev,if (!sock)return -ESHUTDOWN ; - dst =udp_tunnel6_dst_lookup (skb, dev, bareudp->net, sock, + dst =udp_tunnel6_dst_lookup (skb, dev, bareudp->net, sock->sk,0 , &saddr, &info->key, sport, bareudp->port, info->key.tos, use_cache ? &info->dst_cache :NULL );
drivers/net/geneve.c +2 -2
@@ -1424,7 +1424,7 @@ static int geneve6_xmit_skb(struct sk_buff *skb, struct net_device *dev, geneve->cfg.port_min, geneve->cfg.port_max, true); - dst = udp_tunnel6_dst_lookup(skb, dev, geneve->net, gs6->sock, 0, + dst = udp_tunnel6_dst_lookup(skb, dev, geneve->net, gs6->sock->sk, 0, &saddr, key, sport, geneve->cfg.info.key.tp_dst, prio, use_cache ?
@@ -1592,7 +1592,7 @@ static int geneve_fill_metadata_dst(struct net_device *dev, struct sk_buff *skb) geneve->cfg.port_min, geneve->cfg.port_max, true); - dst =udp_tunnel6_dst_lookup (skb, dev, geneve->net, gs6->sock,0 , + dst =udp_tunnel6_dst_lookup (skb, dev, geneve->net, gs6->sock->sk,0 , &saddr, &info->key, sport, geneve->cfg.info.key.tp_dst, prio, use_cache ? &info->dst_cache :NULL );
drivers/net/vxlan/vxlan_core.c +2 -2
@@ -2559,7 +2559,7 @@ void vxlan_xmit_one(struct sk_buff *skb, struct net_device *dev,if (!ifindex) ifindex = sock6->sock->sk->sk_bound_dev_if; - ndst =udp_tunnel6_dst_lookup (skb, dev, vxlan->net, sock6->sock, + ndst =udp_tunnel6_dst_lookup (skb, dev, vxlan->net, sock6->sock->sk, ifindex, &saddr, pkey, src_port, dst_port, tos, use_cache ? dst_cache :NULL );
@@ -3254,7 +3254,7 @@ static int vxlan_fill_metadata_dst(struct net_device *dev, struct sk_buff *skb)if (!sock6)return -EIO ; - ndst = udp_tunnel6_dst_lookup(skb, dev, vxlan->net, sock6->sock, + ndst = udp_tunnel6_dst_lookup(skb, dev, vxlan->net, sock6->sock->sk, 0, &info->key.u.ipv6.src, &info->key, sport, dport, info->key.tos,
include/net/udp_tunnel.h +1 -1
@@ -188,7 +188,7 @@ struct rtable *udp_tunnel_dst_lookup(struct sk_buff *skb, struct dst_entry *udp_tunnel6_dst_lookup(struct sk_buff *skb, struct net_device *dev, struct net *net, - struct socket *sock, int oif, + struct sock *sk, int oif, struct in6_addr *saddr, const struct ip_tunnel_key *key, __be16 sport, __be16 dport, u8 dsfield,
net/ipv6/ip6_udp_tunnel.c +3 -3
@@ -118,7 +118,7 @@ EXPORT_SYMBOL_GPL(udp_tunnel6_xmit_skb);* @skb: Packet for which lookup is done * @dev: Tunnel device * @net: Network namespace of tunnel device -* @sock: Socket which provides route info +* @sk: Socket which provides route info * @oif: Index of the output interface * @saddr: Memory to store the src ip address * @key: Tunnel information
@@ -135,7 +135,7 @@ EXPORT_SYMBOL_GPL(udp_tunnel6_xmit_skb); struct dst_entry *udp_tunnel6_dst_lookup(struct sk_buff *skb, struct net_device *dev, struct net *net, - struct socket *sock, + struct sock *sk, int oif, struct in6_addr *saddr, const struct ip_tunnel_key *key,
@@ -162,7 +162,7 @@ struct dst_entry *udp_tunnel6_dst_lookup(struct sk_buff *skb, fl6.fl6_dport = dport; fl6.flowlabel =ip6_make_flowinfo (dsfield, key->label); - dst =ip6_dst_lookup_flow (net, sock->sk, &fl6,NULL ); + dst =ip6_dst_lookup_flow (net, sk, &fl6,NULL );if (IS_ERR (dst)) {netdev_dbg (dev,"no route to %pI6\n" , &fl6.daddr);return ERR_PTR (-ENETUNREACH );
Signature · 1 line
2.54.0.545.g6539524ca2-goog
PATCH 4/15 udp_tunnel: Pass struct sock to udp_tunnel_{push,drop}_rx_port().
In reply to Kuniyuki IwashimaNone of the udp_tunnel users need struct socket in their fast paths; it is only used for tunnel setup / teardown.
Even udp_tunnel_{push,drop}_rx_port() do not need struct socket.
Let's change udp_tunnel_{push,drop}_rx_port() to take struct sock instead of struct socket.
--- drivers/net/geneve.c | 4 ++-- drivers/net/vxlan/vxlan_core.c | 4 ++-- include/net/udp_tunnel.h | 4 ++-- net/ipv4/udp_tunnel_core.c | 6 ++---- 4 files changed, 8 insertions(+), 10 deletions(-)
drivers/net/geneve.c +2 -2
@@ -1654,10 +1654,10 @@ static void geneve_offload_rx_ports(struct net_device *dev, bool push)list_for_each_entry (gs, &gn->sock_list, list) {if (push) { -udp_tunnel_push_rx_port (dev, gs->sock, +udp_tunnel_push_rx_port (dev, gs->sock->sk,UDP_TUNNEL_TYPE_GENEVE ); }else { -udp_tunnel_drop_rx_port (dev, gs->sock, +udp_tunnel_drop_rx_port (dev, gs->sock->sk,UDP_TUNNEL_TYPE_GENEVE ); } }
drivers/net/vxlan/vxlan_core.c +2 -2
@@ -3332,9 +3332,9 @@ static void vxlan_offload_rx_ports(struct net_device *dev, bool push) type =UDP_TUNNEL_TYPE_VXLAN ;if (push) -udp_tunnel_push_rx_port (dev, vs->sock, type); +udp_tunnel_push_rx_port (dev, vs->sock->sk, type);else -udp_tunnel_drop_rx_port (dev, vs->sock, type); +udp_tunnel_drop_rx_port (dev, vs->sock->sk, type); } } }
include/net/udp_tunnel.h +2 -2
@@ -127,9 +127,9 @@ struct udp_tunnel_info { };/* Notify network devices of offloadable types */ -void udp_tunnel_push_rx_port (struct net_device *dev,struct socket *sock, +void udp_tunnel_push_rx_port (struct net_device *dev,struct sock *sk,unsigned short type); -void udp_tunnel_drop_rx_port (struct net_device *dev,struct socket *sock, +void udp_tunnel_drop_rx_port (struct net_device *dev,struct sock *sk,unsigned short type);void udp_tunnel_notify_add_rx_port (struct socket *sock,unsigned short type);void udp_tunnel_notify_del_rx_port (struct socket *sock,unsigned short type);
net/ipv4/udp_tunnel_core.c +2 -4
@@ -97,10 +97,9 @@ void setup_udp_tunnel_sock(struct net *net, struct sock *sk, }EXPORT_SYMBOL_GPL (setup_udp_tunnel_sock); -void udp_tunnel_push_rx_port (struct net_device *dev,struct socket *sock, +void udp_tunnel_push_rx_port (struct net_device *dev,struct sock *sk,unsigned short type) { -struct sock *sk = sock->sk;struct udp_tunnel_info ti; ti.type = type;
@@ -111,10 +110,9 @@ void udp_tunnel_push_rx_port(struct net_device *dev, struct socket *sock, }EXPORT_SYMBOL_GPL (udp_tunnel_push_rx_port); -void udp_tunnel_drop_rx_port (struct net_device *dev,struct socket *sock, +void udp_tunnel_drop_rx_port (struct net_device *dev,struct sock *sk,unsigned short type) { -struct sock *sk = sock->sk;struct udp_tunnel_info ti; ti.type = type;
Signature · 1 line
2.54.0.545.g6539524ca2-goog
PATCH 5/15 udp_tunnel: Pass struct sock to udp_tunnel_notify_{add,del}_rx_port().
In reply to Kuniyuki IwashimaNone of the udp_tunnel users need struct socket in their fast paths; it is only used for tunnel setup / teardown.
Even udp_tunnel_notify_{add,del}_rx_port() do not need struct socket.
Let's change udp_tunnel_notify_{add,del}_rx_port() to take struct sock instead of struct socket.
--- drivers/net/geneve.c | 4 ++-- drivers/net/vxlan/vxlan_core.c | 4 ++-- include/net/udp_tunnel.h | 4 ++-- net/ipv4/udp_tunnel_core.c | 6 ++---- 4 files changed, 8 insertions(+), 10 deletions(-)
drivers/net/geneve.c +2 -2
@@ -995,7 +995,7 @@ static struct geneve_sock *geneve_socket_create(struct net *net, __be16 port,INIT_HLIST_HEAD (&gs->vni_list[h]);/* Initialize the geneve udp offloads structure */ -udp_tunnel_notify_add_rx_port (gs->sock,UDP_TUNNEL_TYPE_GENEVE ); +udp_tunnel_notify_add_rx_port (gs->sock->sk,UDP_TUNNEL_TYPE_GENEVE );/* Mark socket as an encapsulation socket */ memset (&tunnel_cfg,0 ,sizeof (tunnel_cfg));
@@ -1017,7 +1017,7 @@ static void __geneve_sock_release(struct geneve_sock *gs)return ;list_del (&gs->list); -udp_tunnel_notify_del_rx_port (gs->sock,UDP_TUNNEL_TYPE_GENEVE ); +udp_tunnel_notify_del_rx_port (gs->sock->sk,UDP_TUNNEL_TYPE_GENEVE );udp_tunnel_sock_release (gs->sock->sk);kfree_rcu (gs, rcu); }
drivers/net/vxlan/vxlan_core.c +2 -2
@@ -1493,7 +1493,7 @@ static bool __vxlan_sock_release_prep(struct vxlan_sock *vs)return false;hlist_del_rcu (&vs->hlist); -udp_tunnel_notify_del_rx_port (vs->sock, +udp_tunnel_notify_del_rx_port (vs->sock->sk, (vs->flags &VXLAN_F_GPE ) ?UDP_TUNNEL_TYPE_VXLAN_GPE :UDP_TUNNEL_TYPE_VXLAN );
@@ -3600,7 +3600,7 @@ static struct vxlan_sock *vxlan_socket_create(struct net *net, bool ipv6, vs->flags = (flags &VXLAN_F_RCV_FLAGS );hlist_add_head_rcu (&vs->hlist,vs_head (net, port)); -udp_tunnel_notify_add_rx_port (sock, +udp_tunnel_notify_add_rx_port (sock->sk, (vs->flags &VXLAN_F_GPE ) ?UDP_TUNNEL_TYPE_VXLAN_GPE :UDP_TUNNEL_TYPE_VXLAN );
include/net/udp_tunnel.h +2 -2
@@ -131,8 +131,8 @@ void udp_tunnel_push_rx_port(struct net_device *dev, struct sock *sk, unsigned short type);void udp_tunnel_drop_rx_port (struct net_device *dev,struct sock *sk,unsigned short type); -void udp_tunnel_notify_add_rx_port (struct socket *sock,unsigned short type); -void udp_tunnel_notify_del_rx_port (struct socket *sock,unsigned short type); +void udp_tunnel_notify_add_rx_port (struct sock *sk,unsigned short type); +void udp_tunnel_notify_del_rx_port (struct sock *sk,unsigned short type);/* Transmit the skb using UDP encapsulation. */ void udp_tunnel_xmit_skb(struct rtable *rt, struct sock *sk, struct sk_buff *skb,
net/ipv4/udp_tunnel_core.c +2 -4
@@ -124,9 +124,8 @@ void udp_tunnel_drop_rx_port(struct net_device *dev, struct sock *sk,EXPORT_SYMBOL_GPL (udp_tunnel_drop_rx_port);/* Notify netdevs that UDP port started listening */ -void udp_tunnel_notify_add_rx_port (struct socket *sock,unsigned short type) +void udp_tunnel_notify_add_rx_port (struct sock *sk,unsigned short type) { -struct sock *sk = sock->sk;struct net *net =sock_net (sk);struct udp_tunnel_info ti;struct net_device *dev;
@@ -146,9 +145,8 @@ void udp_tunnel_notify_add_rx_port(struct socket *sock, unsigned short type)EXPORT_SYMBOL_GPL (udp_tunnel_notify_add_rx_port);/* Notify netdevs that UDP port is no more listening */ -void udp_tunnel_notify_del_rx_port (struct socket *sock,unsigned short type) +void udp_tunnel_notify_del_rx_port (struct sock *sk,unsigned short type) { -struct sock *sk = sock->sk;struct net *net =sock_net (sk);struct udp_tunnel_info ti;struct net_device *dev;
Signature · 1 line
2.54.0.545.g6539524ca2-goog
PATCH 6/15 vxlan: Fix potential null-ptr-deref in vxlan_gro_prepare_receive().
In reply to Kuniyuki Iwashimaudp_tunnel_sock_release() could set sk->sk_user_data to NULL while vxlan_gro_prepare_receive() is running.
Let's check if rcu_dereference_sk_user_data() is NULL after skb_gro_remcsum_init().
---
This is 10 years old bug...
--- drivers/net/vxlan/vxlan_core.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
drivers/net/vxlan/vxlan_core.c +6 -2
@@ -657,14 +657,18 @@ static struct vxlanhdr *vxlan_gro_prepare_receive(struct sock *sk,struct sk_buff *skb, struct gro_remcsum *grc) { - struct sk_buff *p; struct vxlanhdr *vh, *vh2;unsigned int hlen, off_vx; -struct vxlan_sock *vs =rcu_dereference_sk_user_data (sk); +struct vxlan_sock *vs; +struct sk_buff *p;__be32 flags;skb_gro_remcsum_init (grc); + vs =rcu_dereference_sk_user_data (sk); +if (!vs) +return NULL ; + off_vx =skb_gro_offset (skb); hlen = off_vx +sizeof (*vh); vh =skb_gro_header (skb, hlen, off_vx);
Signature · 1 line
2.54.0.545.g6539524ca2-goog
PATCH 7/15 vxlan: Store struct sock in struct vxlan_sock.
In reply to Kuniyuki IwashimaCommit 3cf7203ca620 ("net/tunnel: wait until all sk_user_data reader finish before releasing the sock") added synchronize_rcu() in udp_tunnel_sock_release().
This was intended to protect the fast path of a dying vxlan device from dereferencing vxlan_sock->sock->sk after sock_orphan() has set sock->sk to NULL.
However, vxlan does not need to access struct socket itself in the fast path; it only reads struct sock, and struct socket is only used for tunnel setup and teardown.
Let's store struct sock directly in struct vxlan_sock.
In the next patch, we will free vxlan_sock with kfree_rcu(), then vxlan no longer needs synchronize_rcu() in udp_tunnel_sock_release().
--- drivers/net/vxlan/vxlan_core.c | 48 ++++++++++++++--------------- drivers/net/vxlan/vxlan_multicast.c | 8 ++--- include/net/vxlan.h | 4 +-- 3 files changed, 30 insertions(+), 30 deletions(-)
drivers/net/vxlan/vxlan_core.c +24 -24
@@ -88,10 +88,10 @@ static struct vxlan_sock *vxlan_find_sock(struct net *net, sa_family_t family, flags &=VXLAN_F_RCV_FLAGS ;hlist_for_each_entry_rcu (vs,vs_head (net, port), hlist) { -if (inet_sk (vs->sock->sk)->inet_sport == port && +if (inet_sk (vs->sk)->inet_sport == port &&vxlan_get_sk_family (vs) == family && vs->flags == flags && - vs->sock->sk->sk_bound_dev_if == ifindex) + vs->sk->sk_bound_dev_if == ifindex)return vs; }return NULL ;
@@ -1497,7 +1497,7 @@ static bool __vxlan_sock_release_prep(struct vxlan_sock *vs)return false;hlist_del_rcu (&vs->hlist); -udp_tunnel_notify_del_rx_port (vs->sock->sk, +udp_tunnel_notify_del_rx_port (vs->sk, (vs->flags &VXLAN_F_GPE ) ?UDP_TUNNEL_TYPE_VXLAN_GPE :UDP_TUNNEL_TYPE_VXLAN );
@@ -1523,13 +1523,13 @@ static void vxlan_sock_release(struct vxlan_dev *vxlan)vxlan_vs_del_dev (vxlan);if (__vxlan_sock_release_prep (sock4)) { -udp_tunnel_sock_release (sock4->sock->sk); +udp_tunnel_sock_release (sock4->sk);kfree (sock4); }#if IS_ENABLED (CONFIG_IPV6 )if (__vxlan_sock_release_prep (sock6)) { -udp_tunnel_sock_release (sock6->sock->sk); +udp_tunnel_sock_release (sock6->sk);kfree (sock6); }#endif
@@ -2477,7 +2477,7 @@ void vxlan_xmit_one(struct sk_buff *skb, struct net_device *dev, }if (!ifindex) - ifindex = sock4->sock->sk->sk_bound_dev_if; + ifindex = sock4->sk->sk_bound_dev_if; rt = udp_tunnel_dst_lookup(skb, dev, vxlan->net, ifindex, &saddr, pkey, src_port, dst_port,
@@ -2544,7 +2544,7 @@ void vxlan_xmit_one(struct sk_buff *skb, struct net_device *dev, goto tx_error; } -udp_tunnel_xmit_skb (rt, sock4->sock->sk, skb, saddr, +udp_tunnel_xmit_skb (rt, sock4->sk, skb, saddr, pkey->u.ipv4.dst, tos, ttl, df, src_port, dst_port, xnet, !udp_sum, ipcb_flags);
@@ -2561,9 +2561,9 @@ void vxlan_xmit_one(struct sk_buff *skb, struct net_device *dev, }if (!ifindex) - ifindex = sock6->sock->sk->sk_bound_dev_if; + ifindex = sock6->sk->sk_bound_dev_if; - ndst =udp_tunnel6_dst_lookup (skb, dev, vxlan->net, sock6->sock->sk, + ndst =udp_tunnel6_dst_lookup (skb, dev, vxlan->net, sock6->sk, ifindex, &saddr, pkey, src_port, dst_port, tos, use_cache ? dst_cache :NULL );
@@ -2619,7 +2619,7 @@ void vxlan_xmit_one(struct sk_buff *skb, struct net_device *dev, goto tx_error; } -udp_tunnel6_xmit_skb (ndst, sock6->sock->sk, skb, dev, +udp_tunnel6_xmit_skb (ndst, sock6->sk, skb, dev, &saddr, &pkey->u.ipv6.dst, tos, ttl, pkey->label, src_port, dst_port, !udp_sum, ip6cb_flags);
@@ -3258,7 +3258,7 @@ static int vxlan_fill_metadata_dst(struct net_device *dev, struct sk_buff *skb)if (!sock6)return -EIO ; - ndst = udp_tunnel6_dst_lookup(skb, dev, vxlan->net, sock6->sock->sk, + ndst = udp_tunnel6_dst_lookup(skb, dev, vxlan->net, sock6->sk, 0, &info->key.u.ipv6.src, &info->key, sport, dport, info->key.tos,
@@ -3336,9 +3336,9 @@ static void vxlan_offload_rx_ports(struct net_device *dev, bool push) type =UDP_TUNNEL_TYPE_VXLAN ;if (push) -udp_tunnel_push_rx_port (dev, vs->sock->sk, type); +udp_tunnel_push_rx_port (dev, vs->sk, type);else -udp_tunnel_drop_rx_port (dev, vs->sock->sk, type); +udp_tunnel_drop_rx_port (dev, vs->sk, type); } } }
@@ -3544,8 +3544,8 @@ static const struct ethtool_ops vxlan_ethtool_ops = { .get_link_ksettings = vxlan_get_link_ksettings, }; -static struct socket *vxlan_create_sock (struct net *net,bool ipv6, -__be16 port,u32 flags,int ifindex) +static struct sock *vxlan_create_sock (struct net *net,bool ipv6, +__be16 port,u32 flags,int ifindex) {struct socket *sock;struct udp_port_cfg udp_conf;
@@ -3571,7 +3571,7 @@ static struct socket *vxlan_create_sock(struct net *net, bool ipv6,return ERR_PTR (err);udp_allow_gso (sock->sk); -return sock; +return sock->sk; }/* Create new listen socket if needed */
@@ -3579,10 +3579,10 @@ static struct vxlan_sock *vxlan_socket_create(struct net *net, bool ipv6, __be16 port, u32 flags, int ifindex) { + struct udp_tunnel_sock_cfg tunnel_cfg;struct vxlan_sock *vs; -struct socket *sock; +struct sock *sk;unsigned int h; -struct udp_tunnel_sock_cfg tunnel_cfg;ASSERT_RTNL ();
@@ -3593,18 +3593,18 @@ static struct vxlan_sock *vxlan_socket_create(struct net *net, bool ipv6,for (h =0 ; h <VNI_HASH_SIZE ; ++h)INIT_HLIST_HEAD (&vs->vni_list[h]); - sock =vxlan_create_sock (net, ipv6, port, flags, ifindex); -if (IS_ERR (sock)) { + sk =vxlan_create_sock (net, ipv6, port, flags, ifindex); +if (IS_ERR (sk)) {kfree (vs); -return ERR_CAST (sock); +return ERR_CAST (sk); } - vs->sock = sock; + vs->sk = sk;refcount_set (&vs->refcnt,1 ); vs->flags = (flags &VXLAN_F_RCV_FLAGS );hlist_add_head_rcu (&vs->hlist,vs_head (net, port)); -udp_tunnel_notify_add_rx_port (sock->sk, +udp_tunnel_notify_add_rx_port (sk, (vs->flags &VXLAN_F_GPE ) ?UDP_TUNNEL_TYPE_VXLAN_GPE :UDP_TUNNEL_TYPE_VXLAN );
@@ -3624,7 +3624,7 @@ static struct vxlan_sock *vxlan_socket_create(struct net *net, bool ipv6, tunnel_cfg.gro_complete = vxlan_gro_complete; } -setup_udp_tunnel_sock (net, sock->sk, &tunnel_cfg); +setup_udp_tunnel_sock (net, sk, &tunnel_cfg);return vs; }
drivers/net/vxlan/vxlan_multicast.c +4 -4
@@ -29,7 +29,7 @@ int vxlan_igmp_join(struct vxlan_dev *vxlan, union vxlan_addr *rip, .imr_ifindex = ifindex, }; - sk = sock4->sock->sk; + sk = sock4->sk;lock_sock (sk); ret =ip_mc_join_group (sk, &mreq);release_sock (sk);
@@ -37,7 +37,7 @@ int vxlan_igmp_join(struct vxlan_dev *vxlan, union vxlan_addr *rip, }else {struct vxlan_sock *sock6 =rtnl_dereference (vxlan->vn6_sock); - sk = sock6->sock->sk; + sk = sock6->sk;lock_sock (sk); ret =ipv6_sock_mc_join (sk, ifindex, &ip->sin6.sin6_addr);release_sock (sk);
@@ -62,7 +62,7 @@ int vxlan_igmp_leave(struct vxlan_dev *vxlan, union vxlan_addr *rip, .imr_ifindex = ifindex, }; - sk = sock4->sock->sk; + sk = sock4->sk;lock_sock (sk); ret =ip_mc_leave_group (sk, &mreq);release_sock (sk);
@@ -70,7 +70,7 @@ int vxlan_igmp_leave(struct vxlan_dev *vxlan, union vxlan_addr *rip, }else {struct vxlan_sock *sock6 =rtnl_dereference (vxlan->vn6_sock); - sk = sock6->sock->sk; + sk = sock6->sk;lock_sock (sk); ret =ipv6_sock_mc_drop (sk, ifindex, &ip->sin6.sin6_addr);release_sock (sk);
include/net/vxlan.h +2 -2
@@ -185,7 +185,7 @@ struct vxlan_metadata {/* per UDP socket information */ struct vxlan_sock {struct hlist_node hlist; -struct socket *sock; +struct sock *sk;struct hlist_head vni_list[VNI_HASH_SIZE ];refcount_t refcnt;u32 flags;
@@ -448,7 +448,7 @@ static inline __be32 vxlan_compute_rco(unsigned int start, unsigned int offset)static inline unsigned short vxlan_get_sk_family (struct vxlan_sock *vs) { -return vs->sock->sk->sk_family; +return vs->sk->sk_family; }#if IS_ENABLED (CONFIG_IPV6 )
Signature · 1 line
2.54.0.545.g6539524ca2-goog
PATCH 8/15 vxlan: Free vxlan_sock with kfree_rcu().
In reply to Kuniyuki IwashimaWe will remove synchronize_rcu() in udp_tunnel_sock_release().
We must ensure that vxlan_sock is freed after inflight RX fast path.
Let's free vxlan_sock with kfree_rcu().
Note that vxlan_sock.vni_list[] is 8K and struct rcu_head must be placed before it.
--- drivers/net/vxlan/vxlan_core.c | 4 ++-- include/net/vxlan.h | 1 + 2 files changed, 3 insertions(+), 2 deletions(-)
drivers/net/vxlan/vxlan_core.c +2 -2
@@ -1524,13 +1524,13 @@ static void vxlan_sock_release(struct vxlan_dev *vxlan)if (__vxlan_sock_release_prep (sock4)) {udp_tunnel_sock_release (sock4->sk); -kfree (sock4); +kfree_rcu (sock4, rcu); }#if IS_ENABLED (CONFIG_IPV6 )if (__vxlan_sock_release_prep (sock6)) {udp_tunnel_sock_release (sock6->sk); -kfree (sock6); +kfree_rcu (sock6, rcu); }#endif }
include/net/vxlan.h +1 -0
@@ -186,6 +186,7 @@ struct vxlan_metadata {struct vxlan_sock {struct hlist_node hlist;struct sock *sk; +struct rcu_head rcu;struct hlist_head vni_list[VNI_HASH_SIZE ];refcount_t refcnt;u32 flags;
Signature · 1 line
2.54.0.545.g6539524ca2-goog
PATCH 9/15 geneve: Store struct sock in struct geneve_sock.
In reply to Kuniyuki Iwashimageneve does not need to access struct socket itself in the fast path; it only reads struct sock, and struct socket is only used for tunnel setup and teardown.
Let's store struct sock directly in struct geneve_sock.
__geneve_sock_release() frees geneve_sock with kfree_rcu(), so geneve no longer needs synchronize_rcu() in udp_tunnel_sock_release().
--- drivers/net/geneve.c | 54 ++++++++++++++++++++++---------------------- 1 file changed, 27 insertions(+), 27 deletions(-)
drivers/net/geneve.c +27 -27
@@ -106,7 +106,7 @@ struct geneve_sock {bool collect_md;bool gro_hint;struct list_head list; -struct socket *sock; +struct sock *sk;struct rcu_head rcu;int refcnt;struct hlist_head vni_list[VNI_HASH_SIZE ];
@@ -167,7 +167,7 @@ static bool eq_tun_id_and_vni(u8 *tun_id, u8 *vni)static sa_family_t geneve_get_sk_family (struct geneve_sock *gs) { -return gs->sock->sk->sk_family; +return gs->sk->sk_family; } static struct geneve_dev *geneve_lookup(struct geneve_sock *gs,
@@ -760,11 +760,11 @@ static int geneve_udp_encap_err_lookup(struct sock *sk, struct sk_buff *skb)return -EPFNOSUPPORT ; } -static struct socket *geneve_create_sock (struct net *net,bool ipv6, -__be16 port,bool ipv6_rx_csum) +static struct sock *geneve_create_sock (struct net *net,bool ipv6, +__be16 port,bool ipv6_rx_csum) { -struct socket *sock;struct udp_port_cfg udp_conf; +struct socket *sock;int err;memset (&udp_conf,0 ,sizeof (udp_conf));
@@ -786,7 +786,7 @@ static struct socket *geneve_create_sock(struct net *net, bool ipv6,return ERR_PTR (err);udp_allow_gso (sock->sk); -return sock; +return sock->sk; } static bool geneve_hdr_match(struct sk_buff *skb,
@@ -974,28 +974,28 @@ static struct geneve_sock *geneve_socket_create(struct net *net, __be16 port, bool ipv6, bool ipv6_rx_csum) {struct geneve_net *gn =net_generic (net, geneve_net_id); -struct geneve_sock *gs; -struct socket *sock;struct udp_tunnel_sock_cfg tunnel_cfg; +struct geneve_sock *gs; +struct sock *sk;int h; gs =kzalloc_obj (*gs);if (!gs)return ERR_PTR (-ENOMEM ); - sock =geneve_create_sock (net, ipv6, port, ipv6_rx_csum); -if (IS_ERR (sock)) { + sk =geneve_create_sock (net, ipv6, port, ipv6_rx_csum); +if (IS_ERR (sk)) {kfree (gs); -return ERR_CAST (sock); +return ERR_CAST (sk); } - gs->sock = sock; + gs->sk = sk; gs->refcnt =1 ;for (h =0 ; h <VNI_HASH_SIZE ; ++h)INIT_HLIST_HEAD (&gs->vni_list[h]);/* Initialize the geneve udp offloads structure */ -udp_tunnel_notify_add_rx_port (gs->sock->sk,UDP_TUNNEL_TYPE_GENEVE ); +udp_tunnel_notify_add_rx_port (sk,UDP_TUNNEL_TYPE_GENEVE );/* Mark socket as an encapsulation socket */ memset (&tunnel_cfg,0 ,sizeof (tunnel_cfg));
@@ -1006,7 +1006,7 @@ static struct geneve_sock *geneve_socket_create(struct net *net, __be16 port, tunnel_cfg.encap_rcv = geneve_udp_encap_recv; tunnel_cfg.encap_err_lookup = geneve_udp_encap_err_lookup; tunnel_cfg.encap_destroy =NULL ; -setup_udp_tunnel_sock (net, sock->sk, &tunnel_cfg); +setup_udp_tunnel_sock (net, sk, &tunnel_cfg);list_add (&gs->list, &gn->sock_list);return gs; }
@@ -1017,8 +1017,8 @@ static void __geneve_sock_release(struct geneve_sock *gs)return ;list_del (&gs->list); -udp_tunnel_notify_del_rx_port (gs->sock->sk,UDP_TUNNEL_TYPE_GENEVE ); -udp_tunnel_sock_release (gs->sock->sk); +udp_tunnel_notify_del_rx_port (gs->sk,UDP_TUNNEL_TYPE_GENEVE ); +udp_tunnel_sock_release (gs->sk);kfree_rcu (gs, rcu); }
@@ -1048,7 +1048,7 @@ static struct geneve_sock *geneve_find_sock(struct geneve_net *gn,struct geneve_sock *gs;list_for_each_entry (gs, &gn->sock_list, list) { -if (inet_sk (gs->sock->sk)->inet_sport == dst_port && +if (inet_sk (gs->sk)->inet_sport == dst_port &&geneve_get_sk_family (gs) == family && gs->gro_hint == gro_hint) {return gs;
@@ -1390,7 +1390,7 @@ static int geneve_xmit_skb(struct sk_buff *skb, struct net_device *dev,if (unlikely (err))return err; - udp_tunnel_xmit_skb(rt, gs4->sock->sk, skb, saddr, info->key.u.ipv4.dst, + udp_tunnel_xmit_skb(rt, gs4->sk, skb, saddr, info->key.u.ipv4.dst, tos, ttl, df, sport, geneve->cfg.info.key.tp_dst, !net_eq(geneve->net, dev_net(geneve->dev)), !test_bit(IP_TUNNEL_CSUM_BIT, info->key.tun_flags),
@@ -1424,7 +1424,7 @@ static int geneve6_xmit_skb(struct sk_buff *skb, struct net_device *dev, geneve->cfg.port_min, geneve->cfg.port_max, true); - dst = udp_tunnel6_dst_lookup(skb, dev, geneve->net, gs6->sock->sk, 0, + dst = udp_tunnel6_dst_lookup(skb, dev, geneve->net, gs6->sk, 0, &saddr, key, sport, geneve->cfg.info.key.tp_dst, prio, use_cache ?
@@ -1480,7 +1480,7 @@ static int geneve6_xmit_skb(struct sk_buff *skb, struct net_device *dev,if (unlikely (err))return err; -udp_tunnel6_xmit_skb (dst, gs6->sock->sk, skb, dev, +udp_tunnel6_xmit_skb (dst, gs6->sk, skb, dev, &saddr, &key->u.ipv6.dst, prio, ttl, info->key.label, sport, geneve->cfg.info.key.tp_dst, !test_bit(IP_TUNNEL_CSUM_BIT,
@@ -1592,7 +1592,7 @@ static int geneve_fill_metadata_dst(struct net_device *dev, struct sk_buff *skb) geneve->cfg.port_min, geneve->cfg.port_max, true); - dst =udp_tunnel6_dst_lookup (skb, dev, geneve->net, gs6->sock->sk,0 , + dst =udp_tunnel6_dst_lookup (skb, dev, geneve->net, gs6->sk,0 , &saddr, &info->key, sport, geneve->cfg.info.key.tp_dst, prio, use_cache ? &info->dst_cache :NULL );
@@ -1654,10 +1654,10 @@ static void geneve_offload_rx_ports(struct net_device *dev, bool push)list_for_each_entry (gs, &gn->sock_list, list) {if (push) { -udp_tunnel_push_rx_port (dev, gs->sock->sk, +udp_tunnel_push_rx_port (dev, gs->sk,UDP_TUNNEL_TYPE_GENEVE ); }else { -udp_tunnel_drop_rx_port (dev, gs->sock->sk, +udp_tunnel_drop_rx_port (dev, gs->sk,UDP_TUNNEL_TYPE_GENEVE ); } }
@@ -2183,12 +2183,12 @@ static void geneve_quiesce(struct geneve_dev *geneve, struct geneve_sock **gs4, *gs4 =rtnl_dereference (geneve->sock4);rcu_assign_pointer (geneve->sock4,NULL );if (*gs4) -rcu_assign_sk_user_data ((*gs4)->sock->sk,NULL ); +rcu_assign_sk_user_data ((*gs4)->sk,NULL );#if IS_ENABLED (CONFIG_IPV6 ) *gs6 =rtnl_dereference (geneve->sock6);rcu_assign_pointer (geneve->sock6,NULL );if (*gs6) -rcu_assign_sk_user_data ((*gs6)->sock->sk,NULL ); +rcu_assign_sk_user_data ((*gs6)->sk,NULL );#else *gs6 =NULL ;#endif
@@ -2201,11 +2201,11 @@ static void geneve_unquiesce(struct geneve_dev *geneve, struct geneve_sock *gs4, {rcu_assign_pointer (geneve->sock4, gs4);if (gs4) -rcu_assign_sk_user_data (gs4->sock->sk, gs4); +rcu_assign_sk_user_data (gs4->sk, gs4);#if IS_ENABLED (CONFIG_IPV6 )rcu_assign_pointer (geneve->sock6, gs6);if (gs6) -rcu_assign_sk_user_data (gs6->sock->sk, gs6); +rcu_assign_sk_user_data (gs6->sk, gs6);#endif synchronize_net (); }
Signature · 1 line
2.54.0.545.g6539524ca2-goog
PATCH 10/15 bareudp: Store struct sock in struct bareudp_dev.
In reply to Kuniyuki Iwashimabareudp does not need to access struct socket itself in the fast path; it only reads struct sock, and struct socket is only used for tunnel setup and teardown.
Let's store struct sock directly in struct bareudp_dev.
bareudp_sock_release() is called from dev->netdev_ops->ndo_stop(). synchronize_net() in unregister_netdevice_many_notify() ensures that inflight bareudp RX fast paths finish before bareudp_dev is freed.
bareudp no longer needs synchronize_rcu() in udp_tunnel_sock_release().
--- drivers/net/bareudp.c | 51 +++++++++++++++++++++---------------------- 1 file changed, 25 insertions(+), 26 deletions(-)
drivers/net/bareudp.c +25 -26
@@ -53,7 +53,7 @@ struct bareudp_dev {__be16 port;u16 sport_min;bool multi_proto_mode; -struct socket __rcu *sock; +struct sock __rcu *sk;struct list_head next;/* bareudp node on namespace list */ struct gro_cells gro_cells; };
@@ -228,7 +228,7 @@ static void bareudp_uninit(struct net_device *dev)gro_cells_destroy (&bareudp->gro_cells); } -static struct socket *bareudp_create_sock (struct net *net,__be16 port) +static struct sock *bareudp_create_sock (struct net *net,__be16 port) {struct udp_port_cfg udp_conf;struct socket *sock;
@@ -248,18 +248,18 @@ static struct socket *bareudp_create_sock(struct net *net, __be16 port)return ERR_PTR (err);udp_allow_gso (sock->sk); -return sock; +return sock->sk; }/* Create new listen socket if needed */ static int bareudp_socket_create (struct bareudp_dev *bareudp,__be16 port) {struct udp_tunnel_sock_cfg tunnel_cfg; -struct socket *sock; +struct sock *sk; - sock =bareudp_create_sock (bareudp->net, port); -if (IS_ERR (sock)) -return PTR_ERR (sock); + sk =bareudp_create_sock (bareudp->net, port); +if (IS_ERR (sk)) +return PTR_ERR (sk);/* Mark socket as an encapsulation socket */ memset (&tunnel_cfg,0 ,sizeof (tunnel_cfg));
@@ -268,29 +268,27 @@ static int bareudp_socket_create(struct bareudp_dev *bareudp, __be16 port) tunnel_cfg.encap_rcv = bareudp_udp_encap_recv; tunnel_cfg.encap_err_lookup = bareudp_err_lookup; tunnel_cfg.encap_destroy =NULL ; -setup_udp_tunnel_sock (bareudp->net, sock->sk, &tunnel_cfg); +setup_udp_tunnel_sock (bareudp->net, sk, &tunnel_cfg); -rcu_assign_pointer (bareudp->sock, sock); +rcu_assign_pointer (bareudp->sk, sk);return 0 ; }static int bareudp_open (struct net_device *dev) {struct bareudp_dev *bareudp =netdev_priv (dev); -int ret =0 ; - ret =bareudp_socket_create (bareudp, bareudp->port); -return ret; +return bareudp_socket_create (bareudp, bareudp->port); }static void bareudp_sock_release (struct bareudp_dev *bareudp) { -struct socket *sock; +struct sock *sk; - sock = bareudp->sock; -rcu_assign_pointer (bareudp->sock,NULL ); + sk = bareudp->sk; +rcu_assign_pointer (bareudp->sk,NULL );synchronize_net (); -udp_tunnel_sock_release (sock->sk); +udp_tunnel_sock_release (sk); }static int bareudp_stop (struct net_device *dev)
@@ -308,7 +306,7 @@ static int bareudp_xmit_skb(struct sk_buff *skb, struct net_device *dev,bool udp_sum =test_bit (IP_TUNNEL_CSUM_BIT , info->key.tun_flags);bool xnet = !net_eq (bareudp->net,dev_net (bareudp->dev));bool use_cache =ip_tunnel_dst_cache_usable (skb, info); -struct socket *sock =rcu_dereference (bareudp->sock); +struct sock *sk =rcu_dereference (bareudp->sk);const struct ip_tunnel_key *key = &info->key;struct rtable *rt;__be16 sport, df;
@@ -320,7 +318,7 @@ static int bareudp_xmit_skb(struct sk_buff *skb, struct net_device *dev,if (skb_vlan_inet_prepare (skb, skb->protocol !=htons (ETH_P_TEB )))return -EINVAL ; -if (!sock) +if (!sk)return -ESHUTDOWN ; sport = udp_flow_src_port(bareudp->net, skb,
@@ -359,7 +357,7 @@ static int bareudp_xmit_skb(struct sk_buff *skb, struct net_device *dev, goto free_dst;skb_set_inner_protocol (skb, bareudp->ethertype); - udp_tunnel_xmit_skb(rt, sock->sk, skb, saddr, info->key.u.ipv4.dst, + udp_tunnel_xmit_skb(rt, sk, skb, saddr, info->key.u.ipv4.dst, tos, ttl, df, sport, bareudp->port, !net_eq(bareudp->net, dev_net(bareudp->dev)), !test_bit(IP_TUNNEL_CSUM_BIT, info->key.tun_flags),
@@ -378,7 +376,7 @@ static int bareudp6_xmit_skb(struct sk_buff *skb, struct net_device *dev,bool udp_sum =test_bit (IP_TUNNEL_CSUM_BIT , info->key.tun_flags);bool xnet = !net_eq (bareudp->net,dev_net (bareudp->dev));bool use_cache =ip_tunnel_dst_cache_usable (skb, info); -struct socket *sock =rcu_dereference (bareudp->sock); +struct sock *sk =rcu_dereference (bareudp->sk);const struct ip_tunnel_key *key = &info->key;struct dst_entry *dst =NULL ;struct in6_addr saddr, daddr;
@@ -390,13 +388,13 @@ static int bareudp6_xmit_skb(struct sk_buff *skb, struct net_device *dev,if (skb_vlan_inet_prepare (skb, skb->protocol !=htons (ETH_P_TEB )))return -EINVAL ; -if (!sock) +if (!sk)return -ESHUTDOWN ; sport =udp_flow_src_port (bareudp->net, skb, bareudp->sport_min,USHRT_MAX , true); - dst =udp_tunnel6_dst_lookup (skb, dev, bareudp->net, sock->sk,0 , &saddr, + dst =udp_tunnel6_dst_lookup (skb, dev, bareudp->net, sk,0 , &saddr, key, sport, bareudp->port, key->tos, use_cache ? (struct dst_cache *) &info->dst_cache :NULL );
@@ -427,7 +425,7 @@ static int bareudp6_xmit_skb(struct sk_buff *skb, struct net_device *dev, goto free_dst; daddr = info->key.u.ipv6.dst; -udp_tunnel6_xmit_skb (dst, sock->sk, skb, dev, +udp_tunnel6_xmit_skb (dst, sk, skb, dev, &saddr, &daddr, prio, ttl, info->key.label, sport, bareudp->port, !test_bit(IP_TUNNEL_CSUM_BIT,
@@ -527,12 +525,13 @@ static int bareudp_fill_metadata_dst(struct net_device *dev, }else if (ip_tunnel_info_af (info) ==AF_INET6 ) {struct dst_entry *dst;struct in6_addr saddr; -struct socket *sock =rcu_dereference (bareudp->sock); +struct sock *sk; -if (!sock) + sk =rcu_dereference (bareudp->sk); +if (!sk)return -ESHUTDOWN ; - dst =udp_tunnel6_dst_lookup (skb, dev, bareudp->net, sock->sk, + dst =udp_tunnel6_dst_lookup (skb, dev, bareudp->net, sk,0 , &saddr, &info->key, sport, bareudp->port, info->key.tos, use_cache ? &info->dst_cache :NULL );
Signature · 1 line
2.54.0.545.g6539524ca2-goog
Re: [PATCH v1 net-next 10/15] bareudp: Store struct sock in struct bareudp_dev.
For Sashiko's review:
Quoted 5 lines · Kuniyuki Iwashima
bareudp does not need to access struct socket itself in the fast path; it only reads struct sock, and struct socket is only used for tunnel setup and teardown.
Quoted 7 lines
Let's store struct sock directly in struct
bareudp_dev.
bareudp_sock_release()is called fromdev->netdev_ops->ndo_stop().synchronize_net()inunregister_netdevice_many_notify()ensures that inflight bareudp RX fast paths finish beforebareudp_devis freed.bareudp no longer needs
synchronize_rcu()inudp_tunnel_sock_release().
---8<---
While synchronize_net() ensures bareudp_dev is not freed while receive fast paths are inflight, does it protect the underlying UDP socket itself?
...
Since UDP sockets are allocated from a slab cache with SLAB_TYPESAFE_BY_RCU,...
---8<---
Sashiko is lying. UDP doesn't use SLAB_TYPESAFE_BY_RCU, and rather it uses SOCK_RCU_FREE and the socket is freed after RCU grace period.
[...]
drivers/net/bareudp.c static int bareudp_socket_create(struct bareudp_dev *bareudp, __be16 port) · line 292
Quoted 13 lines
static void
bareudp_sock_release(struct bareudp_dev *bareudp){drivers/net/bareudp.c +4
-4-struct socket *sock; +struct sock *sk; - sock = bareudp->sock; -rcu_assign_pointer (bareudp->sock,NULL ); + sk = bareudp->sk; +rcu_assign_pointer (bareudp->sk,NULL );synchronize_net (); -udp_tunnel_sock_release (sock->sk); +udp_tunnel_sock_release (sk); }
---8<---
If an inflight receive packet is preempted right before calling the encap handler, could the socket be freed and reallocated here? ---8<---
No, RTNL is held here, so no one can allocate a new socket.
PATCH 11/15 fou: Store struct sock in struct fou.
In reply to Kuniyuki Iwashimafou does not need to access struct socket itself in the fast path; it only reads struct sock, and struct socket is only used for tunnel setup and teardown.
Let's store struct sock directly in struct fou.
fou_release() frees struct fou with kfree_rcu(), so fou no longer needs synchronize_rcu() in udp_tunnel_sock_release().
Note that the error path in fou_create() looks buggy; once the tunnel is set up and fou_add_to_port_list() fails, struct fou should be freed with kfree_rcu() _after_ udp_tunnel_sock_release().
--- net/ipv4/fou_core.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-)
net/ipv4/fou_core.c +7 -7
@@ -22,7 +22,7 @@#include "fou_nl.h" struct fou { -struct socket *sock; +struct sock *sk;u8 protocol;u8 flags;__be16 port;
@@ -508,8 +508,8 @@ static int gue_gro_complete(struct sock *sk, struct sk_buff *skb, int nhoff)static bool fou_cfg_cmp (struct fou *fou,struct fou_cfg *cfg) { -struct sock *sk = fou->sock->sk;struct udp_port_cfg *udp_cfg = &cfg->udp_config; +struct sock *sk = fou->sk; if (fou->family != udp_cfg->family || fou->port != udp_cfg->local_udp_port ||
@@ -559,7 +559,7 @@ static int fou_add_to_port_list(struct net *net, struct fou *fou,static void fou_release (struct fou *fou) {list_del (&fou->list); -udp_tunnel_sock_release (fou->sock->sk); +udp_tunnel_sock_release (fou->sk);kfree_rcu (fou, rcu); }
@@ -590,7 +590,7 @@ static int fou_create(struct net *net, struct fou_cfg *cfg, fou->family = cfg->udp_config.family; fou->flags = cfg->flags; fou->type = cfg->type; - fou->sock = sock; + fou->sk = sk;memset (&tunnel_cfg,0 ,sizeof (tunnel_cfg)); tunnel_cfg.encap_type =1 ;
@@ -776,9 +776,9 @@ int fou_nl_del_doit(struct sk_buff *skb, struct genl_info *info)static int fou_fill_info (struct fou *fou,struct sk_buff *msg) { -struct sock *sk = fou->sock->sk; +struct sock *sk = fou->sk; - if (nla_put_u8(msg, FOU_ATTR_AF, fou->sock->sk->sk_family) || + if (nla_put_u8(msg, FOU_ATTR_AF, sk->sk_family) || nla_put_be16(msg, FOU_ATTR_PORT, fou->port) || nla_put_be16(msg, FOU_ATTR_PEER_PORT, sk->sk_dport) || nla_put_u8(msg, FOU_ATTR_IPPROTO, fou->protocol) ||
@@ -790,7 +790,7 @@ static int fou_fill_info(struct fou *fou, struct sk_buff *msg)if (nla_put_flag (msg,FOU_ATTR_REMCSUM_NOPARTIAL ))return -1 ; -if (fou->sock->sk->sk_family ==AF_INET ) { +if (sk->sk_family ==AF_INET ) {if (nla_put_in_addr (msg,FOU_ATTR_LOCAL_V4 , sk->sk_rcv_saddr))return -1 ;
Signature · 1 line
2.54.0.545.g6539524ca2-goog
Re: [PATCH v1 net-next 11/15] fou: Store struct sock in struct fou.
For Sashiko review:
Quoted 5 lines · Kuniyuki Iwashima
fou does not need to access struct socket itself in the fast path; it only reads struct sock, and struct socket is only used for tunnel setup and teardown.
Quoted 8 lines
Let's store struct sock directly in struct fou.
fou_release()frees struct fou withkfree_rcu(), so fou no longer needssynchronize_rcu()inudp_tunnel_sock_release().Note that the error path in
fou_create()looks buggy; once the tunnel is set up andfou_add_to_port_list()fails, struct fou should be freed withkfree_rcu()_after_udp_tunnel_sock_release().
---8<---
If this error path is known to be buggy, should it be fixed in this patch or in a preceding patch to avoid a use-after-free?
...
Could this be updated to fix the RCU removal ordering? ---8<---
Yes, but in a separate patch for net. This simple change doesn't even touch the fou_create() error path.
PATCH 12/15 amt: Store struct sock in struct amt_dev.
In reply to Kuniyuki Iwashimaamt does not need to access struct socket itself in the fast path; it only reads struct sock, and struct socket is only used for tunnel setup and teardown.
Let's store struct sock directly in struct amt.
amt_dev_stop() is called as dev->netdev_ops->ndo_stop(). synchronize_net() in unregister_netdevice_many_notify() ensures that inflight amt RX fast paths finish before amt_dev is freed.
amt no longer needs synchronize_rcu() in udp_tunnel_sock_release().
Note that amt_dev_stop() looks buggy; cancel_delayed_work_sync() should be called after udp_tunnel_sock_release().
---
CC: Taehee Yoo <[email protected]>
--- drivers/net/amt.c | 80 +++++++++++++++++++++++------------------------ include/net/amt.h | 2 +- 2 files changed, 41 insertions(+), 41 deletions(-)
drivers/net/amt.c +40 -40
@@ -614,24 +614,24 @@ static void amt_send_discovery(struct amt_dev *amt) {struct amt_header_discovery *amtd;int hlen, tlen, offset; -struct socket *sock;struct udphdr *udph;struct sk_buff *skb;struct iphdr *iph;struct rtable *rt;struct flowi4 fl4; +struct sock *sk;u32 len;int err;rcu_read_lock (); - sock =rcu_dereference (amt->sock); -if (!sock) + sk =rcu_dereference (amt->sk); +if (!sk) goto out;if (!netif_running (amt->stream_dev) || !netif_running (amt->dev)) goto out; - rt = ip_route_output_ports(amt->net, &fl4, sock->sk, + rt = ip_route_output_ports(amt->net, &fl4, sk, amt->discovery_ip, amt->local_ip, amt->gw_port, amt->relay_port, IPPROTO_UDP, 0,
@@ -690,7 +690,7 @@ static void amt_send_discovery(struct amt_dev *amt) skb->ip_summed =CHECKSUM_NONE ;ip_select_ident (amt->net, skb,NULL );ip_send_check (iph); - err =ip_local_out (amt->net, sock->sk, skb); + err =ip_local_out (amt->net, sk, skb);if (unlikely (net_xmit_eval (err))) amt->dev->stats.tx_errors++;
@@ -703,24 +703,24 @@ static void amt_send_request(struct amt_dev *amt, bool v6) {struct amt_header_request *amtrh;int hlen, tlen, offset; -struct socket *sock;struct udphdr *udph;struct sk_buff *skb;struct iphdr *iph;struct rtable *rt;struct flowi4 fl4; +struct sock *sk;u32 len;int err;rcu_read_lock (); - sock =rcu_dereference (amt->sock); -if (!sock) + sk =rcu_dereference (amt->sk); +if (!sk) goto out;if (!netif_running (amt->stream_dev) || !netif_running (amt->dev)) goto out; - rt = ip_route_output_ports(amt->net, &fl4, sock->sk, + rt = ip_route_output_ports(amt->net, &fl4, sk, amt->remote_ip, amt->local_ip, amt->gw_port, amt->relay_port, IPPROTO_UDP, 0,
@@ -781,7 +781,7 @@ static void amt_send_request(struct amt_dev *amt, bool v6) skb->ip_summed =CHECKSUM_NONE ;ip_select_ident (amt->net, skb,NULL );ip_send_check (iph); - err =ip_local_out (amt->net, sock->sk, skb); + err =ip_local_out (amt->net, sk, skb);if (unlikely (net_xmit_eval (err))) amt->dev->stats.tx_errors++;
@@ -1000,14 +1000,14 @@ static bool amt_send_membership_update(struct amt_dev *amt, bool v6) {struct amt_header_membership_update *amtmu; -struct socket *sock;struct iphdr *iph;struct flowi4 fl4;struct rtable *rt; +struct sock *sk;int err; - sock =rcu_dereference_bh (amt->sock); -if (!sock) + sk =rcu_dereference_bh (amt->sk); +if (!sk)return true; err = skb_cow_head(skb, LL_RESERVED_SPACE(amt->dev) + sizeof(*amtmu) +
@@ -1039,7 +1039,7 @@ static bool amt_send_membership_update(struct amt_dev *amt,skb_set_inner_protocol (skb,htons (ETH_P_IP )); elseskb_set_inner_protocol (skb ,htons (ETH_P_IPV6 )); - udp_tunnel_xmit_skb(rt, sock->sk, skb, + udp_tunnel_xmit_skb(rt, sk, skb, fl4.saddr, fl4.daddr, AMT_TOS,
@@ -1060,14 +1060,14 @@ static void amt_send_multicast_data(struct amt_dev *amt, bool v6) {struct amt_header_mcast_data *amtmd; -struct socket *sock;struct sk_buff *skb;struct iphdr *iph;struct flowi4 fl4;struct rtable *rt; +struct sock *sk; - sock =rcu_dereference_bh (amt->sock); -if (!sock) + sk =rcu_dereference_bh (amt->sk); +if (!sk)return ; skb = skb_copy_expand(oskb, sizeof(*amtmd) + sizeof(*iph) +
@@ -1097,7 +1097,7 @@ static void amt_send_multicast_data(struct amt_dev *amt,skb_set_inner_protocol (skb,htons (ETH_P_IP )); elseskb_set_inner_protocol (skb ,htons (ETH_P_IPV6 )); - udp_tunnel_xmit_skb(rt, sock->sk, skb, + udp_tunnel_xmit_skb(rt, sk, skb, fl4.saddr, fl4.daddr, AMT_TOS,
@@ -1116,13 +1116,13 @@ static bool amt_send_membership_query(struct amt_dev *amt, bool v6) {struct amt_header_membership_query *amtmq; -struct socket *sock;struct rtable *rt;struct flowi4 fl4; +struct sock *sk;int err; - sock =rcu_dereference_bh (amt->sock); -if (!sock) + sk =rcu_dereference_bh (amt->sk); +if (!sk)return true; err = skb_cow_head(skb, LL_RESERVED_SPACE(amt->dev) + sizeof(*amtmq) +
@@ -1156,7 +1156,7 @@ static bool amt_send_membership_query(struct amt_dev *amt,skb_set_inner_protocol (skb,htons (ETH_P_IP )); elseskb_set_inner_protocol (skb ,htons (ETH_P_IPV6 )); - udp_tunnel_xmit_skb(rt, sock->sk, skb, + udp_tunnel_xmit_skb(rt, sk, skb, fl4.saddr, fl4.daddr, AMT_TOS,
@@ -2554,24 +2554,24 @@ static void amt_send_advertisement(struct amt_dev *amt, __be32 nonce, {struct amt_header_advertisement *amta;int hlen, tlen, offset; -struct socket *sock;struct udphdr *udph;struct sk_buff *skb;struct iphdr *iph;struct rtable *rt;struct flowi4 fl4; +struct sock *sk;u32 len;int err;rcu_read_lock (); - sock =rcu_dereference (amt->sock); -if (!sock) + sk =rcu_dereference (amt->sk); +if (!sk) goto out;if (!netif_running (amt->stream_dev) || !netif_running (amt->dev)) goto out; - rt = ip_route_output_ports(amt->net, &fl4, sock->sk, + rt = ip_route_output_ports(amt->net, &fl4, sk, daddr, amt->local_ip, dport, amt->relay_port, IPPROTO_UDP, 0,
@@ -2631,7 +2631,7 @@ static void amt_send_advertisement(struct amt_dev *amt, __be32 nonce, skb->ip_summed =CHECKSUM_NONE ;ip_select_ident (amt->net, skb,NULL );ip_send_check (iph); - err =ip_local_out (amt->net, sock->sk, skb); + err =ip_local_out (amt->net, sk, skb);if (unlikely (net_xmit_eval (err))) amt->dev->stats.tx_errors++;
@@ -2944,7 +2944,7 @@ static int amt_err_lookup(struct sock *sk, struct sk_buff *skb)return 0 ; } -static struct socket *amt_create_sock (struct net *net,__be16 port) +static struct sock *amt_create_sock (struct net *net,__be16 port) {struct udp_port_cfg udp_conf;struct socket *sock;
@@ -2960,17 +2960,17 @@ static struct socket *amt_create_sock(struct net *net, __be16 port)if (err <0 )return ERR_PTR (err); -return sock; +return sock->sk; }static int amt_socket_create (struct amt_dev *amt) {struct udp_tunnel_sock_cfg tunnel_cfg; -struct socket *sock; +struct sock *sk; - sock =amt_create_sock (amt->net, amt->relay_port); -if (IS_ERR (sock)) -return PTR_ERR (sock); + sk =amt_create_sock (amt->net, amt->relay_port); +if (IS_ERR (sk)) +return PTR_ERR (sk);/* Mark socket as an encapsulation socket */ memset (&tunnel_cfg,0 ,sizeof (tunnel_cfg));
@@ -2979,9 +2979,9 @@ static int amt_socket_create(struct amt_dev *amt) tunnel_cfg.encap_rcv = amt_rcv; tunnel_cfg.encap_err_lookup = amt_err_lookup; tunnel_cfg.encap_destroy =NULL ; -setup_udp_tunnel_sock (amt->net, sock->sk, &tunnel_cfg); +setup_udp_tunnel_sock (amt->net, sk, &tunnel_cfg); -rcu_assign_pointer (amt->sock, sock); +rcu_assign_pointer (amt->sk, sk);return 0 ; }
@@ -3019,8 +3019,8 @@ static int amt_dev_stop(struct net_device *dev) {struct amt_dev *amt =netdev_priv (dev);struct amt_tunnel_list *tunnel, *tmp; -struct socket *sock;struct sk_buff *skb; +struct sock *sk;int i;cancel_delayed_work_sync (&amt->req_wq);
@@ -3028,11 +3028,11 @@ static int amt_dev_stop(struct net_device *dev)cancel_delayed_work_sync (&amt->secret_wq);/* shutdown */ - sock =rtnl_dereference (amt->sock); -RCU_INIT_POINTER (amt->sock,NULL ); + sk =rtnl_dereference (amt->sk); +RCU_INIT_POINTER (amt->sk,NULL );synchronize_net (); -if (sock) -udp_tunnel_sock_release (sock->sk); +if (sk) +udp_tunnel_sock_release (sk);cancel_work_sync (&amt->event_wq);for (i =0 ; i <AMT_MAX_EVENTS ; i++) {
include/net/amt.h +1 -1
@@ -331,7 +331,7 @@ struct amt_dev {enum amt_status status;/* Generated key */ siphash_key_t key; -struct socket __rcu *sock; +struct sock __rcu *sk;u32 max_groups;u32 max_sources;u32 hash_buckets;
Signature · 1 line
2.54.0.545.g6539524ca2-goog
Re: [PATCH v1 net-next 12/15] amt: Store struct sock in struct amt_dev.
For Sashiko review:
Quoted 7 lines · Kuniyuki Iwashima
amt does not need to access struct socket itself in the fast path; it only reads struct sock, and struct socket is only used for tunnel setup and teardown.
Let's store struct sock directly in struct amt.
Quoted 8 lines
amt_dev_stop()is called asdev->netdev_ops->ndo_stop().synchronize_net()inunregister_netdevice_many_notify()ensures that inflight amt RX fast paths finish beforeamt_devis freed.amt no longer needs
synchronize_rcu()inudp_tunnel_sock_release().Note that
amt_dev_stop()looks buggy;cancel_delayed_work_sync()should be called afterudp_tunnel_sock_release().
---8<---
This isn't a regression introduced by this patch, but if leaving this documented issue unfixed is problematic, should it be addressed here? ---8<---
Pre-existing issue and orthogonal to this series change.
Quoted 7 lines
2 files changed, 41 insertions(+), 41 deletions(-)drivers/net/amt.c +0
-0
drivers/net/amt.c static void amt_send_discovery(struct amt_dev *amt) · line 622
Quoted 10 lines
drivers/net/amt.c +1
-1@@ -614,24 +614,24 @@ static void amt_send_discovery(struct amt_dev *amt) {struct amt_header_discovery *amtd;int hlen, tlen, offset; -struct socket *sock;struct udphdr *udph;struct sk_buff *skb;struct iphdr *iph;struct rtable *rt;struct flowi4 fl4; +struct sock *sk;
---8<---
This isn't a regression introduced by this patch, but does this code miss an SKB headroom reservation here and in multiple other transmit paths?
...
---8<---
Pre-existing issue, and completely unrelated to this series.
PATCH 13/15 pfcp: Store struct sock in struct pfcp_dev.
In reply to Kuniyuki Iwashimapfcp does not need to access struct socket itself in the fast path; it only reads struct sock, and struct socket is only used for tunnel setup and teardown.
Let's store struct sock directly in struct pfcp_dev.
pfcp_del_sock() is called from dev->netdev_ops->ndo_uninit(). The 2nd synchronize_net() in unregister_netdevice_many_notify() ensures that inflight pfcp RX fast paths finish before pfcp_dev is freed.
Note that synchronize_rcu() is added in the error path of pfcp_newlink() since free_netdev() will free pfcp_dev immediately once we remove synchronize_rcu() in udp_tunnel_sock_release().
--- drivers/net/pfcp.c | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-)
drivers/net/pfcp.c +8 -7
@@ -18,7 +18,7 @@struct pfcp_dev {struct list_head list; -struct socket *sock; +struct sock *sk;struct net_device *dev;struct net *net;
@@ -104,8 +104,8 @@ static int pfcp_encap_recv(struct sock *sk, struct sk_buff *skb)static void pfcp_del_sock (struct pfcp_dev *pfcp) { -udp_tunnel_sock_release (pfcp->sock->sk); - pfcp->sock =NULL ; +udp_tunnel_sock_release (pfcp->sk); + pfcp->sk =NULL ; }static void pfcp_dev_uninit (struct net_device *dev)
@@ -151,7 +151,7 @@ static void pfcp_link_setup(struct net_device *dev)netif_keep_dst (dev); } -static struct socket *pfcp_create_sock (struct pfcp_dev *pfcp) +static struct sock *pfcp_create_sock (struct pfcp_dev *pfcp) {struct udp_tunnel_sock_cfg tuncfg = {};struct udp_port_cfg udp_conf = {
@@ -174,14 +174,14 @@ static struct socket *pfcp_create_sock(struct pfcp_dev *pfcp)setup_udp_tunnel_sock (net, sock->sk, &tuncfg); -return sock; +return sock->sk; }static int pfcp_add_sock (struct pfcp_dev *pfcp) { - pfcp->sock =pfcp_create_sock (pfcp); + pfcp->sk =pfcp_create_sock (pfcp); -return PTR_ERR_OR_ZERO (pfcp->sock); +return PTR_ERR_OR_ZERO (pfcp->sk); } static int pfcp_newlink(struct net_device *dev,
@@ -216,6 +216,7 @@ static int pfcp_newlink(struct net_device *dev, exit_del_pfcp_sock:pfcp_del_sock (pfcp); +synchronize_rcu (); exit_err: pfcp->net =NULL ;return err;
Signature · 1 line
2.54.0.545.g6539524ca2-goog
Re: [PATCH v1 net-next 13/15] pfcp: Store struct sock in struct pfcp_dev.
For Sashiko review:
On Fri, May 1, 2026 at 8:14 PM Kuniyuki Iwashima <[email protected]> wrote:
[...]
drivers/net/pfcp.c static int pfcp_newlink(struct net_device *dev, · line 222
Quoted 9 lines
static int pfcp_newlink(struct
net_device*dev,drivers/net/pfcp.c +1
-0@@ -216,6 +216,7 @@ static int pfcp_newlink(struct net_device *dev, exit_del_pfcp_sock:pfcp_del_sock (pfcp); +synchronize_rcu (); exit_err: pfcp->net =NULL ;return err;
---8<---
If register_netdevice() fails after successfully invoking the driver's ndo_init callback, the network core error rollback path executes the ndo_uninit callback, which maps to pfcp_dev_uninit(). pfcp_dev_uninit() safely calls pfcp_del_sock(), which releases the socket and sets pfcp->sk = NULL.
When register_netdevice() propagates the error back, this error handling path unconditionally jumps to exit_del_pfcp_sock and calls pfcp_del_sock() a second time.
Since pfcp_del_sock() lacks a NULL pointer check, it passes the now-NULL pfcp->sk to udp_tunnel_sock_release(), which unconditionally dereferences it:
net/ipv4/udp_tunnel_core.c:udp_tunnel_sock_release() {
struct socket *sock = sk->sk_socket;
...
}
Can this result in a NULL pointer dereference in this error path? Would it make sense to add a check like 'if (!pfcp->sk) return;' inside pfcp_del_sock() to handle this safely?
---8<---
This is pre-existing issue too.
PATCH 14/15 tipc: Store struct sock in struct udp_bearer.
In reply to Kuniyuki Iwashimatipc udp_bearer does not need to access struct socket itself in the fast path; it only reads struct sock, and struct socket is only used for tunnel setup and teardown.
Let's store struct sock directly in struct udp_bearer.
Note that cleanup_bearer() calls synchronize_net() after udp_tunnel_sock_release(), so udp_bearer is not freed until inflight fast paths finish.
Note also that synchronize_rcu() is added in the error path of tipc_udp_enable() since udp_bearer will be kfree()d immediately once we remove synchronize_rcu() in udp_tunnel_sock_release().
This can be later converted to kfree_rcu().
---
Cc: Jon Maloy <[email protected]>
--- net/tipc/udp_media.c | 32 ++++++++++++++++++-------------- 1 file changed, 18 insertions(+), 14 deletions(-)
net/tipc/udp_media.c +18 -14
@@ -89,14 +89,14 @@ struct udp_replicast {/** * struct udp_bearer - ip/udp bearer data structure * @bearer: associated generic tipc bearer -* @ubsock: bearer associated socket +* @sk: bearer associated socket * @ifindex: local address scope * @work: used to schedule deferred work on a bearer * @rcast: associated udp_replicast container */ struct udp_bearer {struct tipc_bearer __rcu *bearer; -struct socket *ubsock; +struct sock *sk;u32 ifindex;struct work_struct work;struct udp_replicast rcast;
@@ -194,7 +194,7 @@ static int tipc_udp_xmit(struct net *net, struct sk_buff *skb, } ttl =ip4_dst_hoplimit (&rt->dst); -udp_tunnel_xmit_skb (rt, ub->ubsock->sk, skb, src->ipv4.s_addr, +udp_tunnel_xmit_skb (rt, ub->sk, skb, src->ipv4.s_addr, dst->ipv4.s_addr,0 , ttl,0 , src->port, dst->port, false, true,0 );#if IS_ENABLED (CONFIG_IPV6 )
@@ -206,7 +206,7 @@ static int tipc_udp_xmit(struct net *net, struct sk_buff *skb, .saddr = src->ipv6, .flowi6_proto = IPPROTO_UDP }; - ndst =ip6_dst_lookup_flow (net, ub->ubsock->sk, + ndst =ip6_dst_lookup_flow (net, ub->sk, &fl6,NULL );if (IS_ERR (ndst)) { err =PTR_ERR (ndst);
@@ -215,7 +215,7 @@ static int tipc_udp_xmit(struct net *net, struct sk_buff *skb, dst_cache_set_ip6(cache, ndst, &fl6.saddr); } ttl =ip6_dst_hoplimit (ndst); -udp_tunnel6_xmit_skb (ndst, ub->ubsock->sk, skb,NULL , +udp_tunnel6_xmit_skb (ndst, ub->sk, skb,NULL , &src->ipv6, &dst->ipv6,0 , ttl,0 , src->port, dst->port, false,0 );#endif
@@ -405,9 +405,9 @@ static int tipc_udp_recv(struct sock *sk, struct sk_buff *skb)static int enable_mcast (struct udp_bearer *ub,struct udp_media_addr *remote) { -int err =0 ; +struct sock *sk = ub->sk;struct ip_mreqn mreqn; -struct sock *sk = ub->ubsock->sk; +int err =0 ;if (ntohs (remote->proto) ==ETH_P_IP ) { mreqn.imr_multiaddr = remote->ipv4;
@@ -670,6 +670,7 @@ static int tipc_udp_enable(struct net *net, struct tipc_bearer *b,struct nlattr *opts[TIPC_NLA_UDP_MAX +1 ];u8 node_id[NODE_ID_LEN ] = {0 ,};struct net_device *dev; +struct socket *sock;int rmcast =0 ; ub =kzalloc_obj (*ub,GFP_ATOMIC );
@@ -764,14 +765,16 @@ static int tipc_udp_enable(struct net *net, struct tipc_bearer *b, goto err; } udp_conf.local_udp_port = local.port; - err =udp_sock_create (net, &udp_conf, &ub->ubsock); + err =udp_sock_create (net, &udp_conf, &sock);if (err) goto err; + + ub->sk = sock->sk; tuncfg.sk_user_data = ub; tuncfg.encap_type =1 ; tuncfg.encap_rcv = tipc_udp_recv; tuncfg.encap_destroy =NULL ; -setup_udp_tunnel_sock (net, ub->ubsock->sk, &tuncfg); +setup_udp_tunnel_sock (net, ub->sk, &tuncfg); err =dst_cache_init (&ub->rcast.dst_cache,GFP_ATOMIC );if (err)
@@ -793,7 +796,8 @@ static int tipc_udp_enable(struct net *net, struct tipc_bearer *b, free:dst_cache_destroy (&ub->rcast.dst_cache); -udp_tunnel_sock_release (ub->ubsock->sk); +udp_tunnel_sock_release (ub->sk); +synchronize_rcu (); err:kfree (ub);return err;
@@ -812,10 +816,10 @@ static void cleanup_bearer(struct work_struct *work)kfree_rcu (rcast, rcu); } - tn =tipc_net (sock_net (ub->ubsock->sk)); + tn =tipc_net (sock_net (ub->sk));dst_cache_destroy (&ub->rcast.dst_cache); -udp_tunnel_sock_release (ub->ubsock->sk); +udp_tunnel_sock_release (ub->sk);/* Note: could use a call_rcu() to avoid another synchronize_net() */ synchronize_net ();
@@ -833,11 +837,11 @@ static void tipc_udp_disable(struct tipc_bearer *b)pr_err ("UDP bearer instance not found\n" );return ; } -sock_set_flag (ub->ubsock->sk,SOCK_DEAD ); +sock_set_flag (ub->sk,SOCK_DEAD );RCU_INIT_POINTER (ub->bearer,NULL );/* sock_release need to be done outside of rtnl lock */ -atomic_inc (&tipc_net (sock_net (ub->ubsock->sk))->wq_count); +atomic_inc (&tipc_net (sock_net (ub->sk))->wq_count);INIT_WORK (&ub->work, cleanup_bearer);schedule_work (&ub->work); }
Signature · 1 line
2.54.0.545.g6539524ca2-goog
Re: [PATCH v1 net-next 14/15] tipc: Store struct sock in struct udp_bearer.
For Sashiko review:
On Fri, May 1, 2026 at 8:14 PM Kuniyuki Iwashima <[email protected]> wrote:
[...]
net/tipc/udp_media.c static int tipc_udp_enable(struct net *net, struct tipc_bearer *b, · line 803
Quoted 10 lines
net/tipc/udp_media.c +2
-1@@ -793,7 +796,8 @@ static int tipc_udp_enable(struct net *net, struct tipc_bearer *b, free:dst_cache_destroy (&ub->rcast.dst_cache); -udp_tunnel_sock_release (ub->ubsock->sk); +udp_tunnel_sock_release (ub->sk); +synchronize_rcu (); err:kfree (ub);return err;
---8<---
Since tipc_udp_enable() is executed under the RTNL mutex, does this explicit synchronize_rcu() cause a system-wide stall for all network configuration operations?
---8<---
Yes, but this is unlikely error path, and as mentioned in the commit message, we can remove this by converting the following kfree() to kfree_rcu().
net/tipc/udp_media.c static void cleanup_bearer(struct work_struct *work) · line 817
Quoted 3 lines
net/tipc/udp_media.c +0
-0@@ -812,10 +816,10 @@ static void cleanup_bearer(struct work_struct *work)kfree_rcu (rcast, rcu); }
---8<---
This is a pre-existing issue, but does calling dst_cache_destroy() before list_del_rcu() violate the rule to remove objects from RCU-protected structures before freeing their resources?
---8<---
Not related to this series.
net/tipc/udp_media.c static void cleanup_bearer(struct work_struct *work) · line 825
Quoted 9 lines
net/tipc/udp_media.c +2
-2- tn =tipc_net (sock_net (ub->ubsock->sk)); + tn =tipc_net (sock_net (ub->sk));dst_cache_destroy (&ub->rcast.dst_cache); -udp_tunnel_sock_release (ub->ubsock->sk); +udp_tunnel_sock_release (ub->sk);/* Note: could use a call_rcu() to avoid another synchronize_net() */ synchronize_net ();
---8<---
Also a pre-existing issue, but since dst_cache_destroy() releases the underlying per-CPU entries via free_percpu() which doesn't wait for RCU readers, does calling it here before synchronize_net() allow RCU readers in tipc_udp_xmit() to concurrently access the freed cache? ---8<---
Same, not related to this series.
PATCH 15/15 udp_tunnel: Remove synchronize_rcu() in udp_tunnel_sock_release().
In reply to Kuniyuki IwashimaCommit 3cf7203ca620 ("net/tunnel: wait until all sk_user_data reader finish before releasing the sock") added synchronize_rcu() in udp_tunnel_sock_release().
This was intended to protect the fast path of a dying vxlan device from dereferencing vxlan_sock->sock->sk after sock_orphan() has set sock->sk to NULL.
However, vxlan does not need to access struct socket itself in the fast path; it only reads struct sock, and struct socket is only used for tunnel setup and teardown.
This applies to all other UDP tunnel users, and they have been converted to access struct sock directly.
In addition, each device-specific struct used in their fast paths is freed after one RCU grace period. Since this occurs after udp_tunnel_sock_release(), the struct is guaranteed to be freed after struct udp_sock.
Therefore, synchronize_rcu() in udp_tunnel_sock_release() is now redundant.
Let's remove it.
Tested:
A script creating/upping vxlan devices in 4000 netns runs 10x faster with this change. We can see the same improvement with other UDP tunnel devices as well.
$ cat vxlan.sh
for i in `seq 1 40`
do
(for j in `seq 1 100` ; do
unshare -n bash -c "ip link add vxlan0 type vxlan id 100 local 127.0.0.1 dstport 4789 && ip link set vxlan0 up";
done) &
done
waitWith bpftrace, we can see vxlan_stop() is significantly faster.
bpftrace -e '
kprobe:vxlan_stop {
@start[tid] = nsecs;
}
kretprobe:vxlan_stop /@start[tid]/ {
@duration_us = hist((nsecs - @start[tid]) / 1000);
delete(@start[tid]);
}
END {
printf("\nExecution time of vxlan_stop (us):\n");
}'Before:
# time ./vxlan.sh // without bpftrace real 0m50.615s user 0m8.171s sys 1m45.101s
@duration_us:
[4K, 8K) 1266 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ | [8K, 16K) 1957 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@| [16K, 32K) 764 |@@@@@@@@@@@@@@@@@@@@ | [32K, 64K) 6 | | [64K, 128K) 4 | | [128K, 256K) 3 | |
After:
# time ./vxlan.sh // without bpftrace real 0m5.247s user 0m7.956s sys 1m47.404s
@duration_us:
[16, 32) 3411 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@| [32, 64) 383 |@@@@@ | [64, 128) 107 |@ | [128, 256) 79 |@ | [256, 512) 16 | | [512, 1K) 2 | | [1K, 2K) 2 | |
Next step is to remove another synchronize_net() in vxlan_stop() and variants in other devices.
--- net/ipv4/udp_tunnel_core.c | 1 - 1 file changed, 1 deletion(-)
net/ipv4/udp_tunnel_core.c +0 -1
@@ -194,7 +194,6 @@ void udp_tunnel_sock_release(struct sock *sk)struct socket *sock = sk->sk_socket;rcu_assign_sk_user_data (sk,NULL ); -synchronize_rcu ();kernel_sock_shutdown (sock,SHUT_RDWR );sock_release (sock); }
Signature · 1 line
2.54.0.545.g6539524ca2-goog
Jakub Kicinski
maintainerQuoted 8 lines · On Sat, 2 May 2026 03:12:53 +0000 Kuniyuki Iwashima wrote:
Kuniyuki Iwashima (15):
udp_tunnel: Pass struct sock toudp_tunnel_sock_release().
udp_tunnel: Pass struct sock tosetup_udp_tunnel_sock().
udp_tunnel: Pass struct sock toudp_tunnel6_dst_lookup().
udp_tunnel: Pass struct sock to udp_tunnel_{push,drop}_rx_port().
udp_tunnel: Pass struct sock to udp_tunnel_notify_{add,del}_rx_port().
vxlan: Fix potential null-ptr-deref invxlan_gro_prepare_receive().
vxlan: Store struct sock in structvxlan_sock.
Quoted 8 lines
vxlan: Free
vxlan_sockwithkfree_rcu().
geneve: Store struct sock in structgeneve_sock.
bareudp: Store struct sock in structbareudp_dev.
fou: Store struct sock in struct fou.
amt: Store struct sock in structamt_dev.
pfcp: Store struct sock in structpfcp_dev.
tipc: Store struct sock in structudp_bearer.
udp_tunnel: Removesynchronize_rcu()inudp_tunnel_sock_release().
Excellent job splitting this into patches, pleasure to read!
Quoted 19 lines · Jakub Kicinski
Quoted 8 lines · On Sat, 2 May 2026 03:12:53 +0000 Kuniyuki Iwashima wrote:
Kuniyuki Iwashima (15):
udp_tunnel: Pass struct sock toudp_tunnel_sock_release().
udp_tunnel: Pass struct sock tosetup_udp_tunnel_sock().
udp_tunnel: Pass struct sock toudp_tunnel6_dst_lookup().
udp_tunnel: Pass struct sock to udp_tunnel_{push,drop}_rx_port().
udp_tunnel: Pass struct sock to udp_tunnel_notify_{add,del}_rx_port().
vxlan: Fix potential null-ptr-deref invxlan_gro_prepare_receive().
vxlan: Store struct sock in structvxlan_sock.Quoted 8 lines
vxlan: Free
vxlan_sockwithkfree_rcu().
geneve: Store struct sock in structgeneve_sock.
bareudp: Store struct sock in structbareudp_dev.
fou: Store struct sock in struct fou.
amt: Store struct sock in structamt_dev.
pfcp: Store struct sock in structpfcp_dev.
tipc: Store struct sock in structudp_bearer.
udp_tunnel: Removesynchronize_rcu()inudp_tunnel_sock_release().
Quoted 1 line
Excellent job splitting this into patches, pleasure to read!
Happy to hear that, thanks ;)
This series was applied to netdev/net-next.git (main) by Jakub Kicinski <[email protected]>:
Hello:
This series was applied to netdev/net-next.git (main) by Jakub Kicinski <[email protected]>:
Quoted 8 lines · On Sat, 2 May 2026 03:12:53 +0000 you wrote:
Most of the UDP tunnel devices call
synchronize_rcu()twice during destruction, for example, vxlan has1)
synchronize_rcu()inudp_tunnel_sock_release()2)
synchronize_net()invxlan_sock_release()[...]
Here is the summary with links:
- [v1,net-next,01/15] udp_tunnel: Pass struct sock to udp_tunnel_sock_release().
https://git.kernel.org/netdev/net-next/c/944bfc1b1c6f - [v1,net-next,02/15] udp_tunnel: Pass struct sock to setup_udp_tunnel_sock(). https://git.kernel.org/netdev/net-next/c/2cba193628fe - [v1,net-next,03/15] udp_tunnel: Pass struct sock to udp_tunnel6_dst_lookup(). https://git.kernel.org/netdev/net-next/c/9333d5ff28bd - [v1,net-next,04/15] udp_tunnel: Pass struct sock to udp_tunnel_{push,drop}_rx_port(). https://git.kernel.org/netdev/net-next/c/8d0012ce1b41 - [v1,net-next,05/15] udp_tunnel: Pass struct sock to udp_tunnel_notify_{add,del}_rx_port(). https://git.kernel.org/netdev/net-next/c/af3f903fbb70 - [v1,net-next,06/15] vxlan: Fix potential null-ptr-deref in vxlan_gro_prepare_receive(). https://git.kernel.org/netdev/net-next/c/30a45c0bffdd - [v1,net-next,07/15] vxlan: Store struct sock in struct vxlan_sock. https://git.kernel.org/netdev/net-next/c/027bffa1ad19 - [v1,net-next,08/15] vxlan: Free vxlan_sock with kfree_rcu(). https://git.kernel.org/netdev/net-next/c/4f71dd1b421d - [v1,net-next,09/15] geneve: Store struct sock in struct geneve_sock. https://git.kernel.org/netdev/net-next/c/82720c6196dc - [v1,net-next,10/15] bareudp: Store struct sock in struct bareudp_dev. https://git.kernel.org/netdev/net-next/c/bc773b15eb19 - [v1,net-next,11/15] fou: Store struct sock in struct fou. https://git.kernel.org/netdev/net-next/c/173a79f6966a - [v1,net-next,12/15] amt: Store struct sock in struct amt_dev. https://git.kernel.org/netdev/net-next/c/65c6e06d5aba - [v1,net-next,13/15] pfcp: Store struct sock in struct pfcp_dev. https://git.kernel.org/netdev/net-next/c/fe57adbe0dac - [v1,net-next,14/15] tipc: Store struct sock in struct udp_bearer. https://git.kernel.org/netdev/net-next/c/1ae552c7b665 - [v1,net-next,15/15] udp_tunnel: Remove synchronize_rcu() in udp_tunnel_sock_release(). https://git.kernel.org/netdev/net-next/c/72d3b9a4c2b1
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html