Skip to content

udp_tunnel: Speed up UDP tunnel device destruction (Part I)

Kuniyuki Iwashima netdev 1 version Applied

Status from this message: This series was applied to netdev/net-next.git (main)

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest15 patches 8 repliesApplied 15/15

Patches in v1 All 15 applied

#Subject and filesSizeRepliesApplied
1/15udp_tunnel: Pass struct sock to udp_tunnel_sock_release().drivers/infiniband/sw/rxe/rxe_net.c drivers/infiniband/sw/rxe/rxe_ns.c drivers/net/amt.c +11 more+27 -281 replyApplied
2/15udp_tunnel: Pass struct sock to setup_udp_tunnel_sock().drivers/infiniband/sw/rxe/rxe_net.c drivers/net/amt.c drivers/net/bareudp.c +13 more+19 -210 repliesApplied
3/15udp_tunnel: Pass struct sock to udp_tunnel6_dst_lookup().drivers/net/bareudp.c drivers/net/geneve.c drivers/net/vxlan/vxlan_core.c +2 more+10 -100 repliesApplied
4/15udp_tunnel: Pass struct sock to udp_tunnel_{push,drop}_rx_port().drivers/net/geneve.c drivers/net/vxlan/vxlan_core.c include/net/udp_tunnel.h +1 more+8 -100 repliesApplied
5/15udp_tunnel: Pass struct sock to udp_tunnel_notify_{add,del}_rx_port().drivers/net/geneve.c drivers/net/vxlan/vxlan_core.c include/net/udp_tunnel.h +1 more+8 -100 repliesApplied
6/15vxlan: Fix potential null-ptr-deref in vxlan_gro_prepare_receive().drivers/net/vxlan/vxlan_core.c+6 -20 repliesApplied
7/15vxlan: Store struct sock in struct vxlan_sock.drivers/net/vxlan/vxlan_core.c drivers/net/vxlan/vxlan_multicast.c include/net/vxlan.h+30 -300 repliesApplied
8/15vxlan: Free vxlan_sock with kfree_rcu().drivers/net/vxlan/vxlan_core.c include/net/vxlan.h+3 -20 repliesApplied
9/15geneve: Store struct sock in struct geneve_sock.drivers/net/geneve.c+27 -270 repliesApplied
10/15bareudp: Store struct sock in struct bareudp_dev.drivers/net/bareudp.c+25 -261 replyApplied
11/15fou: Store struct sock in struct fou.net/ipv4/fou_core.c+7 -71 replyApplied
12/15amt: Store struct sock in struct amt_dev.drivers/net/amt.c include/net/amt.h+41 -411 replyApplied
13/15pfcp: Store struct sock in struct pfcp_dev.drivers/net/pfcp.c+8 -71 replyApplied
14/15tipc: Store struct sock in struct udp_bearer.net/tipc/udp_media.c+18 -141 replyApplied
15/15udp_tunnel: Remove synchronize_rcu() in udp_tunnel_sock_release().net/ipv4/udp_tunnel_core.c+0 -10 repliesApplied

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [PATCH v1 net-next 00/15] udp_tunnel: Speed up UDP tunnel device destruction (Part I)

Most of the UDP tunnel devices call synchronize_rcu() twice during destruction, for example, vxlan has

1) synchronize_rcu() in udp_tunnel_sock_release()

2) synchronize_net() in vxlan_sock_release()

The goal of this series is to remove the former, and another followup series removes the latter.

synchronize_rcu() was added in udp_tunnel_sock_release() by commit 3cf7203ca620 ("net/tunnel: wait until all sk_user_data reader finish before releasing the sock").

This was intended to protect the fast path of a dying vxlan from dereferencing vxlan_sock->sock->sk after sock_orphan() has set sock->sk to NULL.

Most of the UDP tunnel devices store struct socket to its private struct, but it is NOT needed in the fast paths; struct sock is used there, but struct socket is only used for tunnel setup / teardown.

This is probably because UDP tunnel functions accept struct socket, but even such functions do not need it, except for udp_tunnel_sock_release(), which can safely access sk->sk_socket.

The overview of the series:

Patch 1 - 5 : Convert UDP tunnel helper to take struct sock
Patch 6 : Small fix for 10-years-old bug
Patch 7 - 14 : Store struct sock in tunnel devices
Patch 15 : Remove synchronize_rcu() in udp_tunnel_sock_release()

(I noted bugs of fou and amt, so Sashiko may point out
such pre-existing issues in fou, amt, .. and pfcp and tipc, but they are orthogonal and I can follow up separately.)

With this change, a script creating/upping vxlan in 4000 netns runs 10x faster.

  $ cat vxlan.sh
  for i in `seq 1 40`
  do
      (for j in `seq 1 100` ; do
            unshare -n bash -c "ip link add vxlan0 type vxlan id 100 local 127.0.0.1 dstport 4789 && ip link set vxlan0 up";
       done) &
  done
  wait

With bpftrace, we can see vxlan_stop() is significantly faster too.

  bpftrace -e '
  kprobe:vxlan_stop {
          @start[tid] = nsecs;
  }

  kretprobe:vxlan_stop /@start[tid]/ {
          @duration_us = hist((nsecs - @start[tid]) / 1000);
          delete(@start[tid]);
  }

  END {
          printf("\nExecution time of vxlan_stop (us):\n");
  }'

Before:

  # time ./vxlan.sh // without bpftrace
  real  0m50.615s
  user  0m8.171s
  sys   1m45.101s

@duration_us:

  [4K, 8K)            1266 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@                   |
  [8K, 16K)           1957 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@|
  [16K, 32K)           764 |@@@@@@@@@@@@@@@@@@@@                                |
  [32K, 64K)             6 |                                                    |
  [64K, 128K)            4 |                                                    |
  [128K, 256K)           3 |                                                    |

After:

  # time ./vxlan.sh // without bpftrace
  real  0m5.247s
  user  0m7.956s
  sys   1m47.404s

@duration_us:

  [16, 32)            3411 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@|
  [32, 64)             383 |@@@@@                                               |
  [64, 128)            107 |@                                                   |
  [128, 256)            79 |@                                                   |
  [256, 512)            16 |                                                    |
  [512, 1K)              2 |                                                    |
  [1K, 2K)               2 |                                                    |

Kuniyuki Iwashima (15):
udp_tunnel: Pass struct sock to udp_tunnel_sock_release().
udp_tunnel: Pass struct sock to setup_udp_tunnel_sock().
udp_tunnel: Pass struct sock to udp_tunnel6_dst_lookup().
udp_tunnel: Pass struct sock to udp_tunnel_{push,drop}_rx_port().
udp_tunnel: Pass struct sock to udp_tunnel_notify_{add,del}_rx_port().
vxlan: Fix potential null-ptr-deref in vxlan_gro_prepare_receive().
vxlan: Store struct sock in struct vxlan_sock.
vxlan: Free vxlan_sock with kfree_rcu().
geneve: Store struct sock in struct geneve_sock.
bareudp: Store struct sock in struct bareudp_dev.
fou: Store struct sock in struct fou.
amt: Store struct sock in struct amt_dev.
pfcp: Store struct sock in struct pfcp_dev.
tipc: Store struct sock in struct udp_bearer.
udp_tunnel: Remove synchronize_rcu() in udp_tunnel_sock_release().

 drivers/infiniband/sw/rxe/rxe_net.c |  6 +--
 drivers/infiniband/sw/rxe/rxe_ns.c  |  4 +-
 drivers/net/amt.c                   | 80 ++++++++++++++---------------
 drivers/net/bareudp.c               | 51 +++++++++---------
 drivers/net/geneve.c                | 54 +++++++++----------
 drivers/net/gtp.c                   | 10 ++--
 drivers/net/ovpn/udp.c              |  2 +-
 drivers/net/pfcp.c                  | 17 +++---
 drivers/net/vxlan/vxlan_core.c      | 60 ++++++++++++----------
 drivers/net/vxlan/vxlan_multicast.c |  8 +--
 drivers/net/wireguard/socket.c      |  8 +--
 include/net/amt.h                   |  2 +-
 include/net/udp_tunnel.h            | 14 ++---
 include/net/vxlan.h                 |  5 +-
 net/ipv4/fou_core.c                 | 21 ++++----
 net/ipv4/udp_tunnel_core.c          | 23 ++++-----
 net/ipv6/ip6_udp_tunnel.c           |  6 +--
 net/l2tp/l2tp_core.c                |  2 +-
 net/rxrpc/local_object.c            |  2 +-
 net/sctp/protocol.c                 | 10 ++--
 net/tipc/udp_media.c                | 32 +++++++-----
 21 files changed, 209 insertions(+), 208 deletions(-)
Signature · 1 line
2.54.0.545.g6539524ca2-goog