Skip to contentSkip to conversation

bpf, sockmap: Reject sk_msg egress redirects to non-TCP sockets

PATCH Applied Jakub Sitnicki 4 participants 6 messages Sep 20, 2023 – Sep 29, 2023

Subsystems (5)
lore
Share & tools
lore

Jakub Sitnicki

author maintainer
#0 lore raw

PATCH 1/1 bpf, sockmap: Reject sk_msg egress redirects to non-TCP sockets

With a SOCKMAP/SOCKHASH map and an sk_msg program user can steer messages sent from one TCP socket (s1) to actually egress from another TCP socket (s2):

tcp_bpf_sendmsg(s1) // = sk_prot->sendmsg
tcp_bpf_send_verdict(s1) // __SK_REDIRECT case
tcp_bpf_sendmsg_redir(s2)
tcp_bpf_push_locked(s2)
tcp_bpf_push(s2)
tcp_rate_check_app_limited(s2) // expects tcp_sock
tcp_sendmsg_locked(s2) // ditto

There is a hard-coded assumption in the call-chain, that the egress socket (s2) is a TCP socket.

However in commit 122e6c79efe1 ("sock_map: Update sock type checks for UDP") we have enabled redirects to non-TCP sockets. This was done for the sake of BPF sk_skb programs. There was no indention to support sk_msg send-to-egress use case.

As a result, attempts to send-to-egress through a non-TCP socket lead to a crash due to invalid downcast from sock to tcp_sock:

 BUG: kernel NULL pointer dereference, address: 000000000000002f
 ...
 Call Trace:
  <TASK>
  ? show_regs+0x60/0x70
  ? __die+0x1f/0x70
  ? page_fault_oops+0x80/0x160
  ? do_user_addr_fault+0x2d7/0x800
  ? rcu_is_watching+0x11/0x50
  ? exc_page_fault+0x70/0x1c0
  ? asm_exc_page_fault+0x27/0x30
  ? tcp_tso_segs+0x14/0xa0
  tcp_write_xmit+0x67/0xce0
  __tcp_push_pending_frames+0x32/0xf0
  tcp_push+0x107/0x140
  tcp_sendmsg_locked+0x99f/0xbb0
  tcp_bpf_push+0x19d/0x3a0
  tcp_bpf_sendmsg_redir+0x55/0xd0
  tcp_bpf_send_verdict+0x407/0x550
  tcp_bpf_sendmsg+0x1a1/0x390
  inet_sendmsg+0x6a/0x70
  sock_sendmsg+0x9d/0xc0
  ? sockfd_lookup_light+0x12/0x80
  __sys_sendto+0x10e/0x160
  ? syscall_enter_from_user_mode+0x20/0x60
  ? __this_cpu_preempt_check+0x13/0x20
  ? lockdep_hardirqs_on+0x82/0x110
  __x64_sys_sendto+0x1f/0x30
  do_syscall_64+0x38/0x90
  entry_SYSCALL_64_after_hwframe+0x63/0xcd

Reject selecting a non-TCP sockets as redirect target from a BPF sk_msg program to prevent the crash. When attempted, user will receive an EACCES error from send/sendto/sendmsg() syscall.

Fixes 122e6c79efe1 ("sock_map: Update sock type checks for UDP")

---
FYI, I'm working on revamping the sockmap_listen selftest, which exercises some of redirect combinations, to cover the whole combination matrix so that we can catch these kinds of problems early on.

 net/core/sock_map.c | 4 ++++
 1 file changed, 4 insertions(+)
net/core/sock_map.c +4 -0
index cb11750b1df5..4292c2ed1828 100644
@@ -668,6 +668,8 @@ BPF_CALL_4(bpf_msg_redirect_map, struct sk_msg *, msg,
 	sk = __sock_map_lookup_elem(map, key);
 	if (unlikely(!sk || !sock_map_redirect_allowed(sk)))
 		return SK_DROP;
+	if (!(flags & BPF_F_INGRESS) && !sk_is_tcp(sk))
+		return SK_DROP;
 
 	msg->flags = flags;
 	msg->sk_redir = sk;
@@ -1267,6 +1269,8 @@ BPF_CALL_4(bpf_msg_redirect_hash, struct sk_msg *, msg,
 	sk = __sock_hash_lookup_elem(map, key);
 	if (unlikely(!sk || !sock_map_redirect_allowed(sk)))
 		return SK_DROP;
+	if (!(flags & BPF_F_INGRESS) && !sk_is_tcp(sk))
+		return SK_DROP;
 
 	msg->flags = flags;
 	msg->sk_redir = sk;
Signature · 1 line
2.41.0
#1 lore raw
Quoted 15 lines · Jakub Sitnicki
net/core/sock_map.c +2 -0
index cb11750b1df5..4292c2ed1828 100644
@@ -668,6 +668,8 @@ BPF_CALL_4(bpf_msg_redirect_map, struct sk_msg *, msg,
  	sk = __sock_map_lookup_elem(map, key);
  	if (unlikely(!sk || !sock_map_redirect_allowed(sk)))
  		return SK_DROP;
+	if (!(flags & BPF_F_INGRESS) && !sk_is_tcp(sk))
+		return SK_DROP;
 
  	msg->flags = flags;
  	msg->sk_redir = sk;

net/core/sock_map.c BPF_CALL_4(bpf_msg_redirect_hash, struct sk_msg *, msg, · line 1276

Quoted 8 lines
net/core/sock_map.c +2 -0
@@ -1267,6 +1269,8 @@ BPF_CALL_4(bpf_msg_redirect_hash, struct sk_msg *, msg,
  	sk = __sock_hash_lookup_elem(map, key);
  	if (unlikely(!sk || !sock_map_redirect_allowed(sk)))
  		return SK_DROP;
+	if (!(flags & BPF_F_INGRESS) && !sk_is_tcp(sk))
+		return SK_DROP;
 
  	msg->flags = flags;
  	msg->sk_redir = sk;

Just be curious! Can it happen to other socket types?
I mean to redirect a msg from a sk of any type to one of another type.

author maintainer
#2 lore raw
Quoted 16 lines · Kui-Feng Lee
Quoted 15 lines · Jakub Sitnicki
net/core/sock_map.c +2 -0
index cb11750b1df5..4292c2ed1828 100644
@@ -668,6 +668,8 @@ BPF_CALL_4(bpf_msg_redirect_map, struct sk_msg *, msg,
  	sk = __sock_map_lookup_elem(map, key);
  	if (unlikely(!sk || !sock_map_redirect_allowed(sk)))
  		return SK_DROP;
+	if (!(flags & BPF_F_INGRESS) && !sk_is_tcp(sk))
+		return SK_DROP;
    	msg->flags = flags;
  	msg->sk_redir = sk;
@@ -1267,6 +1269,8 @@ BPF_CALL_4(bpf_msg_redirect_hash, struct sk_msg *, msg,
  	sk = __sock_hash_lookup_elem(map, key);
  	if (unlikely(!sk || !sock_map_redirect_allowed(sk)))
Quoted 8 lines

net/core/sock_map.c BPF_CALL_4(bpf_msg_redirect_map, struct sk_msg *, msg, · line 675

Quoted 5 lines
net/core/sock_map.c +2 -0
  		return SK_DROP;
+	if (!(flags & BPF_F_INGRESS) && !sk_is_tcp(sk))
+		return SK_DROP;
    	msg->flags = flags;
  	msg->sk_redir = sk;

Just be curious! Can it happen to other socket types?
I mean to redirect a msg from a sk of any type to one of another type.

Today sk_msg redirects are implemented only for tcp4 and tcp6.

Here's a full matrix of what redirects are supported [1].

[1] https://gist.github.com/jsitnicki/578fdd614d181bed2b02922b17972b4e

#3 lore raw
Quoted 26 lines · Jakub Sitnicki
Quoted 16 lines · Kui-Feng Lee
Quoted 15 lines · Jakub Sitnicki
net/core/sock_map.c +2 -0
index cb11750b1df5..4292c2ed1828 100644
@@ -668,6 +668,8 @@ BPF_CALL_4(bpf_msg_redirect_map, struct sk_msg *, msg,
   	sk = __sock_map_lookup_elem(map, key);
   	if (unlikely(!sk || !sock_map_redirect_allowed(sk)))
   		return SK_DROP;
+	if (!(flags & BPF_F_INGRESS) && !sk_is_tcp(sk))
+		return SK_DROP;
     	msg->flags = flags;
   	msg->sk_redir = sk;
@@ -1267,6 +1269,8 @@ BPF_CALL_4(bpf_msg_redirect_hash, struct sk_msg *, msg,
   	sk = __sock_hash_lookup_elem(map, key);
   	if (unlikely(!sk || !sock_map_redirect_allowed(sk)))
Quoted 8 lines

net/core/sock_map.c BPF_CALL_4(bpf_msg_redirect_map, struct sk_msg *, msg, · line 675

Quoted 5 lines
net/core/sock_map.c +2 -0
   		return SK_DROP;
+	if (!(flags & BPF_F_INGRESS) && !sk_is_tcp(sk))
+		return SK_DROP;
     	msg->flags = flags;
   	msg->sk_redir = sk;

Just be curious! Can it happen to other socket types?
I mean to redirect a msg from a sk of any type to one of another type.

Quoted 5 lines

Today sk_msg redirects are implemented only for tcp4 and tcp6.

Here's a full matrix of what redirects are supported [1].

[1] https://gist.github.com/jsitnicki/578fdd614d181bed2b02922b17972b4e

Thanks!

maintainer Acked-by on v1 1/1
#4 lore raw
In reply to Jakub Sitnicki
Quoted 58 lines · Jakub Sitnicki

With a SOCKMAP/SOCKHASH map and an sk_msg program user can steer messages sent from one TCP socket (s1) to actually egress from another TCP socket (s2):

tcp_bpf_sendmsg(s1) // = sk_prot->sendmsg
tcp_bpf_send_verdict(s1) // __SK_REDIRECT case
tcp_bpf_sendmsg_redir(s2)
tcp_bpf_push_locked(s2)
tcp_bpf_push(s2)
tcp_rate_check_app_limited(s2) // expects tcp_sock
tcp_sendmsg_locked(s2) // ditto

There is a hard-coded assumption in the call-chain, that the egress socket (s2) is a TCP socket.

However in commit 122e6c79efe1 ("sock_map: Update sock type checks for UDP") we have enabled redirects to non-TCP sockets. This was done for the sake of BPF sk_skb programs. There was no indention to support sk_msg send-to-egress use case.

As a result, attempts to send-to-egress through a non-TCP socket lead to a crash due to invalid downcast from sock to tcp_sock:

 BUG: kernel NULL pointer dereference, address: 000000000000002f
 ...
 Call Trace:
  <TASK>
  ? show_regs+0x60/0x70
  ? __die+0x1f/0x70
  ? page_fault_oops+0x80/0x160
  ? do_user_addr_fault+0x2d7/0x800
  ? rcu_is_watching+0x11/0x50
  ? exc_page_fault+0x70/0x1c0
  ? asm_exc_page_fault+0x27/0x30
  ? tcp_tso_segs+0x14/0xa0
  tcp_write_xmit+0x67/0xce0
  __tcp_push_pending_frames+0x32/0xf0
  tcp_push+0x107/0x140
  tcp_sendmsg_locked+0x99f/0xbb0
  tcp_bpf_push+0x19d/0x3a0
  tcp_bpf_sendmsg_redir+0x55/0xd0
  tcp_bpf_send_verdict+0x407/0x550
  tcp_bpf_sendmsg+0x1a1/0x390
  inet_sendmsg+0x6a/0x70
  sock_sendmsg+0x9d/0xc0
  ? sockfd_lookup_light+0x12/0x80
  __sys_sendto+0x10e/0x160
  ? syscall_enter_from_user_mode+0x20/0x60
  ? __this_cpu_preempt_check+0x13/0x20
  ? lockdep_hardirqs_on+0x82/0x110
  __x64_sys_sendto+0x1f/0x30
  do_syscall_64+0x38/0x90
  entry_SYSCALL_64_after_hwframe+0x63/0xcd

Reject selecting a non-TCP sockets as redirect target from a BPF sk_msg program to prevent the crash. When attempted, user will receive an EACCES error from send/sendto/sendmsg() syscall.

Quoted 6 lines
Fixes 122e6c79efe1 ("sock_map: Update sock type checks for UDP")

---
FYI, I'm working on revamping the sockmap_listen selftest, which exercises some of redirect combinations, to cover the whole combination matrix so that we can catch these kinds of problems early on.

Yes this would be appreciated.

Quoted 18 lines
 net/core/sock_map.c | 4 ++++
 1 file changed, 4 insertions(+)
net/core/sock_map.c +2 -0
index cb11750b1df5..4292c2ed1828 100644
@@ -668,6 +668,8 @@ BPF_CALL_4(bpf_msg_redirect_map, struct sk_msg *, msg,
 	sk = __sock_map_lookup_elem(map, key);
 	if (unlikely(!sk || !sock_map_redirect_allowed(sk)))
 		return SK_DROP;
+	if (!(flags & BPF_F_INGRESS) && !sk_is_tcp(sk))
+		return SK_DROP;
 
 	msg->flags = flags;
 	msg->sk_redir = sk;

net/core/sock_map.c BPF_CALL_4(bpf_msg_redirect_hash, struct sk_msg *, msg, · line 1273

Quoted 5 lines
net/core/sock_map.c +2 -0
@@ -1267,6 +1269,8 @@ BPF_CALL_4(bpf_msg_redirect_hash, struct sk_msg *, msg,
 	sk = __sock_hash_lookup_elem(map, key);
 	if (unlikely(!sk || !sock_map_redirect_allowed(sk)))
 		return SK_DROP;
+	if (!(flags & BPF_F_INGRESS) && !sk_is_tcp(sk))
+		return SK_DROP;

As a stop gap I think this is fine. If anyone wants to add support though I do think as a use case it would make sense to redirect TCP into an AF_UNIX socket and vice versa.

#5 lore raw
In reply to Jakub Sitnicki
This patch was applied to bpf/bpf.git (master) by Daniel Borkmann <[email protected]>:

Hello:

This patch was applied to bpf/bpf.git (master)
by Daniel Borkmann <[email protected]>:

Quoted 12 lines · On Wed, 20 Sep 2023 12:20:55 +0200 you wrote:

With a SOCKMAP/SOCKHASH map and an sk_msg program user can steer messages sent from one TCP socket (s1) to actually egress from another TCP socket (s2):

tcp_bpf_sendmsg(s1) // = sk_prot->sendmsg
tcp_bpf_send_verdict(s1) // __SK_REDIRECT case
tcp_bpf_sendmsg_redir(s2)
tcp_bpf_push_locked(s2)
tcp_bpf_push(s2)
tcp_rate_check_app_limited(s2) // expects tcp_sock
tcp_sendmsg_locked(s2) // ditto

Quoted 1 line

[...]

Here is the summary with links:
- [bpf] bpf, sockmap: Reject sk_msg egress redirects to non-TCP sockets

    https://git.kernel.org/bpf/bpf/c/b80e31baa436

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html