net: retrieve netns cookie via getsocketopt
Subsystems (9)
PATCH 1/2 net: retrieve netns cookie via getsocketopt
It's getting more common to run nested container environments for testing cloud software. One of such examples is Kind [1] which runs a Kubernetes cluster in Docker containers on a single host. Each container acts as a Kubernetes node, and thus can run any Pod (aka container) inside the former. This approach simplifies testing a lot, as it eliminates complicated VM setups.
Unfortunately, such a setup breaks some functionality when cgroupv2 BPF programs are used for load-balancing. The load-balancer BPF program needs to detect whether a request originates from the host netns or a container netns in order to allow some access, e.g. to a service via a loopback IP address. Typically, the programs detect this by comparing netns cookies with the one of the init ns via a call to bpf_get_netns_cookie(NULL). However, in nested environments the latter cannot be used given the Kubernetes node's netns is outside the init ns. To fix this, we need to pass the Kubernetes node netns cookie to the program in a different way: by extending getsockopt() with a SO_NETNS_COOKIE option, the orchestrator which runs in the Kubernetes node netns can retrieve the cookie and pass it to the program instead.
Thus, this is following up on Eric's commit 3d368ab87cf6 ("net:
initialize net->net_cookie at netns setup") to allow retrieval via SO_NETNS_COOKIE. This is also in line in how we retrieve socket cookie via SO_COOKIE.
--- arch/alpha/include/uapi/asm/socket.h | 2 ++ arch/mips/include/uapi/asm/socket.h | 2 ++ arch/parisc/include/uapi/asm/socket.h | 2 ++ arch/sparc/include/uapi/asm/socket.h | 2 ++ include/uapi/asm-generic/socket.h | 2 ++ net/core/sock.c | 9 +++++++++ 6 files changed, 19 insertions(+)
arch/alpha/include/uapi/asm/socket.h +2 -0
@@ -127,6 +127,8 @@#define SO_PREFER_BUSY_POLL 69#define SO_BUSY_POLL_BUDGET 70 +#define SO_NETNS_COOKIE 71 +#if !defined(__KERNEL__)#if __BITS_PER_LONG ==64
arch/mips/include/uapi/asm/socket.h +2 -0
@@ -138,6 +138,8 @@#define SO_PREFER_BUSY_POLL 69#define SO_BUSY_POLL_BUDGET 70 +#define SO_NETNS_COOKIE 71 +#if !defined(__KERNEL__)#if __BITS_PER_LONG ==64
arch/parisc/include/uapi/asm/socket.h +2 -0
@@ -119,6 +119,8 @@#define SO_PREFER_BUSY_POLL 0x4043#define SO_BUSY_POLL_BUDGET 0x4044 +#define SO_NETNS_COOKIE 0x4045 +#if !defined(__KERNEL__)#if __BITS_PER_LONG ==64
arch/sparc/include/uapi/asm/socket.h +2 -0
@@ -120,6 +120,8 @@#define SO_PREFER_BUSY_POLL 0x0048#define SO_BUSY_POLL_BUDGET 0x0049 +#define SO_NETNS_COOKIE 0x0050 +#if !defined(__KERNEL__)
include/uapi/asm-generic/socket.h +2 -0
@@ -122,6 +122,8 @@#define SO_PREFER_BUSY_POLL 69#define SO_BUSY_POLL_BUDGET 70 +#define SO_NETNS_COOKIE 71 +#if !defined(__KERNEL__)#if __BITS_PER_LONG ==64 || (defined(__x86_64__) && defined(__ILP32__))
net/core/sock.c +9 -0
@@ -1635,6 +1635,15 @@ int sock_getsockopt(struct socket *sock, int level, int optname, v.val = sk->sk_bound_dev_if;break ; +#ifdef CONFIG_NET_NS +case SO_NETNS_COOKIE : + lv =sizeof (u64); +if (len != lv) +return -EINVAL ; + v.val64 =sock_net (sk)->net_cookie; +break ; +#endif +default :/* We implement the SO_SNDLOWAT etc to not be settable * (1003.1g 7).
Signature · 1 line
2.32.0
PATCH 2/2 tools/testing: add a selftest for SO_NETNS_COOKIE
From: Lorenz Bauer <[email protected]>
Make sure that SO_NETNS_COOKIE returns a non-zero value, and that sockets from different namespaces have a distinct cookie value.
--- tools/testing/selftests/net/.gitignore | 1 + tools/testing/selftests/net/Makefile | 2 +- tools/testing/selftests/net/config | 1 + tools/testing/selftests/net/so_netns_cookie.c | 61 +++++++++++++++++++ 4 files changed, 64 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/net/so_netns_cookie.c
tools/testing/selftests/net/.gitignore +1 -0
@@ -30,3 +30,4 @@ hwtstamp_config rxtimestamp timestamping txtimestamp +so_netns_cookie
tools/testing/selftests/net/Makefile +1 -1
@@ -30,7 +30,7 @@ TEST_GEN_FILES = socket nettest TEST_GEN_FILES += psock_fanout psock_tpacket msg_zerocopy reuseport_addr_any TEST_GEN_FILES += tcp_mmap tcp_inq psock_snd txring_overwrite TEST_GEN_FILES += udpgso udpgso_bench_tx udpgso_bench_rx ip_defrag -TEST_GEN_FILES += so_txtime ipv6_flowlabel ipv6_flowlabel_mgr +TEST_GEN_FILES += so_txtime ipv6_flowlabel ipv6_flowlabel_mgr so_netns_cookie TEST_GEN_FILES += tcp_fastopen_backup_key TEST_GEN_FILES += fin_ack_lat TEST_GEN_FILES += reuseaddr_ports_exhausted
tools/testing/selftests/net/config +1 -0
@@ -1,4 +1,5 @@ CONFIG_USER_NS=y +CONFIG_NET_NS=y CONFIG_BPF_SYSCALL=y CONFIG_TEST_BPF=m CONFIG_NUMA=y
tools/testing/selftests/net/so_netns_cookie.c new file +61 -0
61 lines not on this page · raw message
Signature · 1 line
2.32.0
Quoted 110 lines · Martynas Pumputis
It's getting more common to run nested container environments for testing cloud software. One of such examples is Kind [1] which runs a Kubernetes cluster in Docker containers on a single host. Each container acts as a Kubernetes node, and thus can run any Pod (aka container) inside the former. This approach simplifies testing a lot, as it eliminates complicated VM setups.
Unfortunately, such a setup breaks some functionality when cgroupv2 BPF programs are used for load-balancing. The load-balancer BPF program needs to detect whether a request originates from the host netns or a container netns in order to allow some access, e.g. to a service via a loopback IP address. Typically, the programs detect this by comparing netns cookies with the one of the init ns via a call to
bpf_get_netns_cookie(NULL). However, in nested environments the latter cannot be used given the Kubernetes node's netns is outside the init ns. To fix this, we need to pass the Kubernetes node netns cookie to the program in a different way: by extendinggetsockopt()with aSO_NETNS_COOKIEoption, the orchestrator which runs in the Kubernetes node netns can retrieve the cookie and pass it to the program instead.Thus, this is following up on Eric's commit
3d368ab87cf6("net:
initializenet->net_cookieat netns setup") to allow retrieval viaSO_NETNS_COOKIE. This is also in line in how we retrieve socket cookie viaSO_COOKIE.Signed-off-by Lorenz Bauer <[email protected]>Signed-off-by Martynas Pumputis <[email protected]>Cc Eric Dumazet <[email protected]>--- arch/alpha/include/uapi/asm/socket.h | 2 ++ arch/mips/include/uapi/asm/socket.h | 2 ++ arch/parisc/include/uapi/asm/socket.h | 2 ++ arch/sparc/include/uapi/asm/socket.h | 2 ++ include/uapi/asm-generic/socket.h | 2 ++ net/core/sock.c | 9 +++++++++ 6 files changed, 19 insertions(+)arch/alpha/include/uapi/asm/socket.h +2
-0@@ -127,6 +127,8 @@#define SO_PREFER_BUSY_POLL 69#define SO_BUSY_POLL_BUDGET 70 +#define SO_NETNS_COOKIE 71 +#if !defined(__KERNEL__)#if __BITS_PER_LONG ==64 arch/mips/include/uapi/asm/socket.h +2
-0@@ -138,6 +138,8 @@#define SO_PREFER_BUSY_POLL 69#define SO_BUSY_POLL_BUDGET 70 +#define SO_NETNS_COOKIE 71 +#if !defined(__KERNEL__)#if __BITS_PER_LONG ==64 arch/parisc/include/uapi/asm/socket.h +2
-0@@ -119,6 +119,8 @@#define SO_PREFER_BUSY_POLL 0x4043#define SO_BUSY_POLL_BUDGET 0x4044 +#define SO_NETNS_COOKIE 0x4045 +#if !defined(__KERNEL__)#if __BITS_PER_LONG ==64 arch/sparc/include/uapi/asm/socket.h +2
-0@@ -120,6 +120,8 @@#define SO_PREFER_BUSY_POLL 0x0048#define SO_BUSY_POLL_BUDGET 0x0049 +#define SO_NETNS_COOKIE 0x0050 +#if !defined(__KERNEL__)include/uapi/asm-generic/socket.h +2
-0@@ -122,6 +122,8 @@#define SO_PREFER_BUSY_POLL 69#define SO_BUSY_POLL_BUDGET 70 +#define SO_NETNS_COOKIE 71 +#if !defined(__KERNEL__)#if __BITS_PER_LONG ==64 || (defined(__x86_64__) && defined(__ILP32__))net/core/sock.c +0
-0
net/core/sock.c int sock_getsockopt(struct socket *sock, int level, int optname, · line 1638
Quoted 4 lines
net/core/sock.c +1
-0@@ -1635,6 +1635,15 @@ int sock_getsockopt(struct socket *sock, int level, int optname, v.val = sk->sk_bound_dev_if;break ; +#ifdef CONFIG_NET_NS
Nit: sock_net already takes care of CONFIG_NET_NS and returns the root ns if !NET_NS, so this define is not necessary. I think this behaviour is nicer: uapi stays consistent even if kernel config changes.
Quoted 14 lines
net/core/sock.c +8
-1+case SO_NETNS_COOKIE : + lv =sizeof (u64); +if (len != lv) +return -EINVAL ; + v.val64 =sock_net (sk)->net_cookie; +break ; +#endif +default :/* We implement the SO_SNDLOWAT etc to not be settable * (1003.1g 7). -- 2.32.0
Signature · 4 lines
Lorenz Bauer | Systems Engineer 6th Floor, County Hall/The Riverside Building, SE1 7PB, UK www.cloudflare.com