Skip to contentSkip to conversation

bpf: Reject TCP_NODELAY in TCP header option callbacks

PATCH Superseded KaFai Wan 5 participants 13 messages Apr 17 – Apr 21

Subsystems (6)

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore

KaFai Wan

author
#0 lore raw

v3 cover bpf: Reject TCP_NODELAY in TCP header option callbacks

This small patchset is about avoid infinite recursion in TCP header option callbacks via TCP_NODELAY setsockopt.

v3:
- Remove CONFIG_INET check and add comment (Martin and Jiayuan)
- Fix the test case (Martin)

v2:
- Reject TCP_NODELAY in bpf_sock_ops_setsockopt() (AI and Martin)
https://lore.kernel.org/bpf/[email protected]/

v1:
https://lore.kernel.org/bpf/[email protected]/

---
KaFai Wan (2):
bpf: Reject TCP_NODELAY in TCP header option callbacks selftests/bpf: Test TCP_NODELAY in TCP hdr opt callbacks

 net/core/filter.c                                 |  6 ++++++
 .../selftests/bpf/prog_tests/tcp_hdr_options.c    | 12 +++++++++++-
 .../bpf/progs/test_misc_tcp_hdr_options.c         | 15 ++++++++++++++-
 3 files changed, 31 insertions(+), 2 deletions(-)
Signature · 1 line
2.43.0
author
#1 lore raw

PATCH v3 1/2 bpf: Reject TCP_NODELAY in TCP header option callbacks

A BPF_SOCK_OPS program can enable BPF_SOCK_OPS_WRITE_HDR_OPT_CB_FLAG and then call bpf_setsockopt(TCP_NODELAY) from BPF_SOCK_OPS_HDR_OPT_LEN_CB or BPF_SOCK_OPS_WRITE_HDR_OPT_CB.

In these callbacks, bpf_setsockopt(TCP_NODELAY) can reach __tcp_sock_set_nodelay(), which can call tcp_push_pending_frames().

From BPF_SOCK_OPS_HDR_OPT_LEN_CB, tcp_push_pending_frames() can call tcp_current_mss(), which calls tcp_established_options() and re-enters bpf_skops_hdr_opt_len().

BPF_SOCK_OPS_HDR_OPT_LEN_CB
-> bpf_setsockopt(TCP_NODELAY)

    -> tcp_push_pending_frames()
      -> tcp_current_mss()
        -> tcp_established_options()
          -> bpf_skops_hdr_opt_len()
            -> BPF_SOCK_OPS_HDR_OPT_LEN_CB

From BPF_SOCK_OPS_WRITE_HDR_OPT_CB, tcp_push_pending_frames() can call tcp_write_xmit(), which calls tcp_transmit_skb(). That path recomputes header option length through tcp_established_options() and bpf_skops_hdr_opt_len() before re-entering bpf_skops_write_hdr_opt().

BPF_SOCK_OPS_WRITE_HDR_OPT_CB
-> bpf_setsockopt(TCP_NODELAY)

    -> tcp_push_pending_frames()
      -> tcp_write_xmit()
        -> tcp_transmit_skb()
          -> tcp_established_options()
            -> bpf_skops_hdr_opt_len()
          -> bpf_skops_write_hdr_opt()
            -> BPF_SOCK_OPS_WRITE_HDR_OPT_CB

This leads to unbounded recursion and can overflow the kernel stack.

Reject TCP_NODELAY with -EOPNOTSUPP in bpf_sock_ops_setsockopt() when bpf_setsockopt() is called from BPF_SOCK_OPS_HDR_OPT_LEN_CB or BPF_SOCK_OPS_WRITE_HDR_OPT_CB.

Fixes 7e41df5dbba2 ("bpf: Add a few optnames to bpf_setsockopt")
---
 net/core/filter.c | 6 ++++++
 1 file changed, 6 insertions(+)
net/core/filter.c +6 -0
index 5fa9189eb772..96849f4c1fbc 100644
@@ -5833,6 +5833,12 @@ BPF_CALL_5(bpf_sock_ops_setsockopt, struct bpf_sock_ops_kern *, bpf_sock,
 	if (!is_locked_tcp_sock_ops(bpf_sock))
 		return -EOPNOTSUPP;
 
+	/* TCP_NODELAY triggers tcp_push_pending_frames() and re-enters these callbacks. */
+	if ((bpf_sock->op == BPF_SOCK_OPS_HDR_OPT_LEN_CB ||
+	     bpf_sock->op == BPF_SOCK_OPS_WRITE_HDR_OPT_CB) &&
+	    level == SOL_TCP && optname == TCP_NODELAY)
+		return -EOPNOTSUPP;
+
 	return _bpf_setsockopt(bpf_sock->sk, level, optname, optval, optlen);
 }
 
Signature · 1 line
2.43.0
author
#8 lore raw

PATCH v3 2/2 selftests/bpf: Test TCP_NODELAY in TCP hdr opt callbacks

In reply to KaFai Wan

Add a sockops selftest for the TCP_NODELAY restriction in BPF_SOCK_OPS_HDR_OPT_LEN_CB and BPF_SOCK_OPS_WRITE_HDR_OPT_CB.

With BPF_SOCK_OPS_WRITE_HDR_OPT_CB_FLAG enabled, bpf_setsockopt(TCP_NODELAY) returns -EOPNOTSUPP from BPF_SOCK_OPS_HDR_OPT_LEN_CB and BPF_SOCK_OPS_WRITE_HDR_OPT_CB, avoiding unbounded recursion and kernel stack overflow.

Other cases continue to work as before, including BPF_SOCK_OPS_PASSIVE_ESTABLISHED_CB and user space setsockopt(TCP_NODELAY).

---
 .../selftests/bpf/prog_tests/tcp_hdr_options.c    | 12 +++++++++++-
 .../bpf/progs/test_misc_tcp_hdr_options.c         | 15 ++++++++++++++-
 2 files changed, 25 insertions(+), 2 deletions(-)
tools/testing/selftests/bpf/prog_tests/tcp_hdr_options.c +11 -1
index 56685fc03c7e..7b9dbbb84316 100644
@@ -461,7 +461,7 @@ static void misc(void)
 	const unsigned int nr_data = 2;
 	struct bpf_link *link;
 	struct sk_fds sk_fds;
-	int i, ret;
+	int i, ret, true_val = 1;
 
 	lport_linum_map_fd = bpf_map__fd(misc_skel->maps.lport_linum_map);
 
@@ -477,6 +477,10 @@ static void misc(void)
 		return;
 	}
 
+	ret = setsockopt(sk_fds.active_fd, SOL_TCP, TCP_NODELAY, &true_val, sizeof(true_val));
+	if (!ASSERT_OK(ret, "setsockopt(TCP_NODELAY)"))
+		goto check_linum;
+
 	for (i = 0; i < nr_data; i++) {
 		/* MSG_EOR to ensure skb will not be combined */
 		ret = send(sk_fds.active_fd, send_msg, sizeof(send_msg),
@@ -507,6 +511,12 @@ static void misc(void)
 
 	ASSERT_EQ(misc_skel->bss->nr_hwtstamp, 0, "nr_hwtstamp");
 
+	ASSERT_TRUE(misc_skel->data->nodelay_est_ok, "unexpected nodelay_est_ok");
+
+	ASSERT_TRUE(misc_skel->data->nodelay_hdr_len_err, "unexpected nodelay_hdr_len_err");
+
+	ASSERT_TRUE(misc_skel->data->nodelay_write_hdr_err, "unexpected nodelay_write_hdr_err");
+
 check_linum:
 	ASSERT_FALSE(check_error_linum(&sk_fds), "check_error_linum");
 	sk_fds_close(&sk_fds);
tools/testing/selftests/bpf/progs/test_misc_tcp_hdr_options.c +14 -1
index d487153a839d..a02e28d9db2e 100644
@@ -29,6 +29,10 @@ unsigned int nr_syn = 0;
 unsigned int nr_fin = 0;
 unsigned int nr_hwtstamp = 0;
 
+bool nodelay_est_ok = true;
+bool nodelay_hdr_len_err = true;
+bool nodelay_write_hdr_err = true;
+
 /* Check the header received from the active side */
 static int __check_active_hdr_in(struct bpf_sock_ops *skops, bool check_syn)
 {
@@ -300,7 +304,7 @@ static int handle_passive_estab(struct bpf_sock_ops *skops)
 SEC("sockops")
 int misc_estab(struct bpf_sock_ops *skops)
 {
-	int true_val = 1;
+	int true_val = 1, false_val = 0, ret;
 
 	switch (skops->op) {
 	case BPF_SOCK_OPS_TCP_LISTEN_CB:
@@ -316,10 +320,19 @@ int misc_estab(struct bpf_sock_ops *skops)
 	case BPF_SOCK_OPS_PARSE_HDR_OPT_CB:
 		return handle_parse_hdr(skops);
 	case BPF_SOCK_OPS_HDR_OPT_LEN_CB:
+		ret = bpf_setsockopt(skops, SOL_TCP, TCP_NODELAY, &true_val, sizeof(true_val));
+		nodelay_hdr_len_err &= ret == -EOPNOTSUPP;
+
 		return handle_hdr_opt_len(skops);
 	case BPF_SOCK_OPS_WRITE_HDR_OPT_CB:
+		ret = bpf_setsockopt(skops, SOL_TCP, TCP_NODELAY, &true_val, sizeof(true_val));
+		nodelay_write_hdr_err &= ret == -EOPNOTSUPP;
+
 		return handle_write_hdr_opt(skops);
 	case BPF_SOCK_OPS_PASSIVE_ESTABLISHED_CB:
+		ret = bpf_setsockopt(skops, SOL_TCP, TCP_NODELAY, &false_val, sizeof(false_val));
+		nodelay_est_ok &= ret == 0;
+
 		return handle_passive_estab(skops);
 	}
 
Signature · 1 line
2.43.0
#2 lore raw
In reply to KaFai Wan
Does this fix also need to block TCP_CORK? When bpf_setsockopt() is called with SOL_TCP and TCP_CORK where the value is 0 (disabling cork), the call path goes through:
Quoted 5 lines
net/core/filter.c +0 -0
index 5fa9189eb772..96849f4c1fbc 100644

net/core/filter.c BPF_CALL_5(bpf_sock_ops_setsockopt, struct bpf_sock_ops_kern *, bpf_sock, · line 5841

Quoted 9 lines
net/core/filter.c +6 -0
@@ -5833,6 +5833,12 @@ BPF_CALL_5(bpf_sock_ops_setsockopt, struct bpf_sock_ops_kern *, bpf_sock,
 	if (!is_locked_tcp_sock_ops(bpf_sock))
 		return -EOPNOTSUPP;
 
+	/* TCP_NODELAY triggers tcp_push_pending_frames() and re-enters these callbacks. */
+	if ((bpf_sock->op == BPF_SOCK_OPS_HDR_OPT_LEN_CB ||
+	     bpf_sock->op == BPF_SOCK_OPS_WRITE_HDR_OPT_CB) &&
+	    level == SOL_TCP && optname == TCP_NODELAY)
+		return -EOPNOTSUPP;
+

Does this fix also need to block TCP_CORK? When bpf_setsockopt() is called with SOL_TCP and TCP_CORK where the value is 0 (disabling cork), the call path goes through:

bpf_sock_ops_setsockopt()
-> _bpf_setsockopt()

    -> __bpf_setsockopt()
      -> sol_tcp_sockopt()
        -> do_tcp_setsockopt()
          -> __tcp_sock_set_cork(sk, false)

In __tcp_sock_set_cork() at net/ipv4/tcp.c, when cork is disabled, it unconditionally calls tcp_push_pending_frames(sk):

net/ipv4/tcp.c:__tcp_sock_set_cork() {
    ...
    tcp->nonagle |= TCP_NAGLE_OFF|TCP_NAGLE_PUSH;

    tcp_push_pending_frames(sk);
}

This triggers the same recursion chain described in the commit message:

tcp_push_pending_frames()
-> tcp_current_mss()

    -> tcp_established_options()
      -> bpf_skops_hdr_opt_len()
        -> BPF_CGROUP_RUN_PROG_SOCK_OPS_SK (re-enters with
           op=BPF_SOCK_OPS_HDR_OPT_LEN_CB)

causing unbounded recursion and kernel stack overflow. Note that enabling TCP_CORK (val=1) is safe since it doesn't call tcp_push_pending_frames(), only disabling it (val=0) does.

net/core/filter.c BPF_CALL_5(bpf_sock_ops_setsockopt, struct bpf_sock_ops_kern *, bpf_sock, · line 5843

Quoted 2 lines
net/core/filter.c +0 -0
 	return _bpf_setsockopt(bpf_sock->sk, level, optname, optval, optlen);
 }

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/24558395229

Reviewed-by on v3 1/2
#3 lore raw
In reply to KaFai Wan
Quoted 41 lines · KaFai Wan

A BPF_SOCK_OPS program can enable BPF_SOCK_OPS_WRITE_HDR_OPT_CB_FLAG and then call bpf_setsockopt(TCP_NODELAY) from BPF_SOCK_OPS_HDR_OPT_LEN_CB or BPF_SOCK_OPS_WRITE_HDR_OPT_CB.

In these callbacks, bpf_setsockopt(TCP_NODELAY) can reach __tcp_sock_set_nodelay(), which can call tcp_push_pending_frames().

From BPF_SOCK_OPS_HDR_OPT_LEN_CB, tcp_push_pending_frames() can call
tcp_current_mss(), which calls tcp_established_options() and re-enters bpf_skops_hdr_opt_len().

BPF_SOCK_OPS_HDR_OPT_LEN_CB
-> bpf_setsockopt(TCP_NODELAY)

     -> tcp_push_pending_frames()
       -> tcp_current_mss()
         -> tcp_established_options()
           -> bpf_skops_hdr_opt_len()
             -> BPF_SOCK_OPS_HDR_OPT_LEN_CB

From BPF_SOCK_OPS_WRITE_HDR_OPT_CB, tcp_push_pending_frames() can call
tcp_write_xmit(), which calls tcp_transmit_skb(). That path recomputes header option length through tcp_established_options() and bpf_skops_hdr_opt_len() before re-entering bpf_skops_write_hdr_opt().

BPF_SOCK_OPS_WRITE_HDR_OPT_CB
-> bpf_setsockopt(TCP_NODELAY)

     -> tcp_push_pending_frames()
       -> tcp_write_xmit()
         -> tcp_transmit_skb()
           -> tcp_established_options()
             -> bpf_skops_hdr_opt_len()
           -> bpf_skops_write_hdr_opt()
             -> BPF_SOCK_OPS_WRITE_HDR_OPT_CB

This leads to unbounded recursion and can overflow the kernel stack.

Reject TCP_NODELAY with -EOPNOTSUPP in bpf_sock_ops_setsockopt() when bpf_setsockopt() is called from BPF_SOCK_OPS_HDR_OPT_LEN_CB or BPF_SOCK_OPS_WRITE_HDR_OPT_CB.

Quoted 6 lines
Fixes 7e41df5dbba2 ("bpf: Add a few optnames to bpf_setsockopt")

I think the result of AI review is false-positve.

Reviewed-by on v3 2/2
#9 lore raw

Re: [PATCH bpf v3 2/2] selftests/bpf: Test TCP_NODELAY in TCP hdr opt callbacks

In reply to KaFai Wan
Quoted 8 lines · KaFai Wan

Add a sockops selftest for the TCP_NODELAY restriction in BPF_SOCK_OPS_HDR_OPT_LEN_CB and BPF_SOCK_OPS_WRITE_HDR_OPT_CB.

With BPF_SOCK_OPS_WRITE_HDR_OPT_CB_FLAG enabled, bpf_setsockopt(TCP_NODELAY) returns -EOPNOTSUPP from BPF_SOCK_OPS_HDR_OPT_LEN_CB and BPF_SOCK_OPS_WRITE_HDR_OPT_CB, avoiding unbounded recursion and kernel stack overflow.

Quoted 5 lines

Other cases continue to work as before, including BPF_SOCK_OPS_PASSIVE_ESTABLISHED_CB and user space setsockopt(TCP_NODELAY).

A little nit below, no need to resend.

Quoted 10 lines
---
  .../selftests/bpf/prog_tests/tcp_hdr_options.c    | 12 +++++++++++-
  .../bpf/progs/test_misc_tcp_hdr_options.c         | 15 ++++++++++++++-
  2 files changed, 25 insertions(+), 2 deletions(-)
tools/testing/selftests/bpf/prog_tests/tcp_hdr_options.c +0 -0
index 56685fc03c7e..7b9dbbb84316 100644

tools/testing/selftests/bpf/prog_tests/tcp_hdr_options.c static void misc(void) · line 465

Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/tcp_hdr_options.c +1 -1
@@ -461,7 +461,7 @@ static void misc(void)
  	const unsigned int nr_data = 2;
  	struct bpf_link *link;
  	struct sk_fds sk_fds;
-	int i, ret;
+	int i, ret, true_val = 1;
 

NIT: please follow the reverse xmas tree variable ordering

Quoted 14 lines
tools/testing/selftests/bpf/prog_tests/tcp_hdr_options.c +4 -0
  	lport_linum_map_fd = bpf_map__fd(misc_skel->maps.lport_linum_map);
 
@@ -477,6 +477,10 @@ static void misc(void)
  		return;
  	}
 
+	ret = setsockopt(sk_fds.active_fd, SOL_TCP, TCP_NODELAY, &true_val, sizeof(true_val));
+	if (!ASSERT_OK(ret, "setsockopt(TCP_NODELAY)"))
+		goto check_linum;
+
  	for (i = 0; i < nr_data; i++) {
  		/* MSG_EOR to ensure skb will not be combined */
  		ret = send(sk_fds.active_fd, send_msg, sizeof(send_msg),

tools/testing/selftests/bpf/prog_tests/tcp_hdr_options.c static void misc(void) · line 519

Quoted 9 lines
tools/testing/selftests/bpf/prog_tests/tcp_hdr_options.c +6 -0
@@ -507,6 +511,12 @@ static void misc(void)
 
  	ASSERT_EQ(misc_skel->bss->nr_hwtstamp, 0, "nr_hwtstamp");
 
+	ASSERT_TRUE(misc_skel->data->nodelay_est_ok, "unexpected nodelay_est_ok");
+
+	ASSERT_TRUE(misc_skel->data->nodelay_hdr_len_err, "unexpected nodelay_hdr_len_err");
+
+	ASSERT_TRUE(misc_skel->data->nodelay_write_hdr_err, "unexpected nodelay_write_hdr_err");
+

NIT: It's would be misleading if you run ./test_progs with "-v" misc:PASS:unexpected nodelay_est_ok 0 nsec

"PASS:unexpected" ?

Quoted 8 lines
tools/testing/selftests/bpf/prog_tests/tcp_hdr_options.c +0 -0
  check_linum:
  	ASSERT_FALSE(check_error_linum(&sk_fds), "check_error_linum");
  	sk_fds_close(&sk_fds);
tools/testing/selftests/bpf/progs/test_misc_tcp_hdr_options.c +0 -0
index d487153a839d..a02e28d9db2e 100644

tools/testing/selftests/bpf/progs/test_misc_tcp_hdr_options.c unsigned int nr_syn = 0; · line 34

Quoted 6 lines
tools/testing/selftests/bpf/progs/test_misc_tcp_hdr_options.c +3 -0
@@ -29,6 +29,10 @@ unsigned int nr_syn = 0;
  unsigned int nr_fin = 0;
  unsigned int nr_hwtstamp = 0;
 
+bool nodelay_est_ok = true;
+bool nodelay_hdr_len_err = true;
+bool nodelay_write_hdr_err = true;

I prefer "nodelay_hdr_len_reject"

#10 lore raw

Re: [PATCH bpf v3 2/2] selftests/bpf: Test TCP_NODELAY in TCP hdr opt callbacks

In reply to KaFai Wan
Quoted 14 lines · KaFai Wan
tools/testing/selftests/bpf/prog_tests/tcp_hdr_options.c +1 -1
index 56685fc03c7e..7b9dbbb84316 100644
@@ -461,7 +461,7 @@ static void misc(void)
 	const unsigned int nr_data = 2;
 	struct bpf_link *link;
 	struct sk_fds sk_fds;
-	int i, ret;
+	int i, ret, true_val = 1;
 
 	lport_linum_map_fd = bpf_map__fd(misc_skel->maps.lport_linum_map);
 

tools/testing/selftests/bpf/prog_tests/tcp_hdr_options.c static void misc(void) · line 480

Quoted 4 lines
tools/testing/selftests/bpf/prog_tests/tcp_hdr_options.c +1 -0
@@ -477,6 +477,10 @@ static void misc(void)
 		return;
 	}
 
+	ret = setsockopt(sk_fds.active_fd, SOL_TCP, TCP_NODELAY, &true_val, sizeof(true_val));

Same comment as in v2. Why this setsockopt is needed? The setsockopt in userspace is unnecessary. In the future, we may need to understand why it is needed here in the first place.

author
#11 lore raw

Re: [PATCH bpf v3 2/2] selftests/bpf: Test TCP_NODELAY in TCP hdr opt callbacks

Quoted 14 lines · Martin KaFai Lau
Quoted 13 lines · KaFai Wan
tools/testing/selftests/bpf/prog_tests/tcp_hdr_options.c +1 -1
index 56685fc03c7e..7b9dbbb84316 100644
@@ -461,7 +461,7 @@ static void misc(void)
  	const unsigned int nr_data = 2;
  	struct bpf_link *link;
  	struct sk_fds sk_fds;
-	int i, ret;
+	int i, ret, true_val = 1;


  lport_linum_map_fd = bpf_map__fd(misc_skel->maps.lport_linum_map);

Quoted 8 lines

tools/testing/selftests/bpf/prog_tests/tcp_hdr_options.c static void misc(void) · line 480

Quoted 6 lines

tools/testing/selftests/bpf/prog_tests/tcp_hdr_options.c +0 -0
@@ -477,6 +477,10 @@ static void misc(void)
  		return;
  	}

tools/testing/selftests/bpf/prog_tests/tcp_hdr_options.c +1 -0
+	ret = setsockopt(sk_fds.active_fd, SOL_TCP, TCP_NODELAY, &true_val, sizeof(true_val));

Same comment as in v2. Why this setsockopt is needed?

Sorry I miss this. It's from the review of v1, my first version would break the syscall setsockopt and other CB besides HDR_OPT_LEN/WRITE_HDR_OPT. So in the test I check setsockopt() and bpf_setsockopt() in PASSIVE_ESTABLISHED_CB to make sure patch#1 would not break user space and other CB.

Quoted 1 line

The setsockopt in userspace is unnecessary.

Is bpf_setsockopt() in PASSIVE_ESTABLISHED_CB also unnecessary? I'll respin if they are unnecessary.

Quoted 2 lines

In the future,
we may need to understand why it is needed here in the first place.

Okay, I'll remember that. Thanks for the review and guidance.

Signature · 2 lines
Thanks,
KaFai
#4 lore raw
In reply to KaFai Wan
Sashiko AI review found 1 potential issue(s): - [High] The patch prevents an infinite recursion causing a kernel stack overflow by rejecting TCP_NODELAY in specific BPF_SOCK_OPS callbacks. However…

Sashiko AI review found 1 potential issue(s):
- [High] The patch prevents an infinite recursion causing a kernel stack overflow by rejecting TCP_NODELAY in specific BPF_SOCK_OPS callbacks. However, it fails to address the exact same recursion vulnerability achievable via bpf_sk_setsockopt in a BPF TCP congestion control algorithm (BPF_PROG_TYPE_STRUCT_OPS). --

commit 4ff4d98f6b4e3c3d90f9268761de84573f49fd86
Author: KaFai Wan <[email protected]>

bpf: Reject TCP_NODELAY in TCP header option callbacks

This patch rejects bpf_setsockopt(TCP_NODELAY) when called from BPF_SOCK_OPS_HDR_OPT_LEN_CB or BPF_SOCK_OPS_WRITE_HDR_OPT_CB to prevent unbounded recursion and kernel stack overflow.

net/core/filter.c BPF_CALL_5(bpf_sock_ops_setsockopt, struct bpf_sock_ops_kern *, bpf_sock, · line 5840

Quoted 13 lines
net/core/filter.c +5 -0
index fcfcb72663ca3..eae2f2c542716 100644
@@ -5833,6 +5833,12 @@ BPF_CALL_5(bpf_sock_ops_setsockopt, struct bpf_sock_ops_kern *, bpf_sock,
 	if (!is_locked_tcp_sock_ops(bpf_sock))
 		return -EOPNOTSUPP;
 
+	/* TCP_NODELAY triggers tcp_push_pending_frames() and re-enters these callbacks. */
+	if ((bpf_sock->op == BPF_SOCK_OPS_HDR_OPT_LEN_CB ||
+	     bpf_sock->op == BPF_SOCK_OPS_WRITE_HDR_OPT_CB) &&
+	    level == SOL_TCP && optname == TCP_NODELAY)
+		return -EOPNOTSUPP;

Does this same recursion vulnerability exist for BPF TCP congestion control algorithms using BPF_PROG_TYPE_STRUCT_OPS?

If a BPF congestion control algorithm invokes bpf_setsockopt(TCP_NODELAY) from its cwnd_event callback when handling CA_EVENT_TX_START, could it trigger the same unbounded recursion?

When the kernel transmits the first packet of a data train via tcp_transmit_skb(), it invokes tcp_event_data_sent(). Because tp->packets_out is not incremented until later, tcp_packets_in_flight(tp) evaluates to 0, triggering tcp_ca_event(sk, CA_EVENT_TX_START).

If the BPF program then calls bpf_setsockopt(TCP_NODELAY), it would result in this call chain:

tcp_transmit_skb()
tcp_event_data_sent() -> invokes CA_EVENT_TX_START
cwnd_event()
bpf_setsockopt(TCP_NODELAY)
tcp_push_pending_frames()
tcp_write_xmit()

Since the outer tcp_transmit_skb() hasn't finished, the send head hasn't advanced. Wouldn't tcp_write_xmit() see the same SKB, attempt to transmit it again, and re-enter tcp_transmit_skb() causing an infinite recursion?

Should the restriction on TCP_NODELAY be enforced at a broader level, such as inside _bpf_setsockopt(), to protect contexts holding the socket lock during TX paths?

Quoted 4 lines
net/core/filter.c +1 -0
+
 	return _bpf_setsockopt(bpf_sock->sk, level, optname, optval, optlen);
 }
 
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
author
#5 lore raw
Quoted 54 lines · On Sat, 2026-04-18 at 09:22 +0000, [email protected] wrote:

Sashiko AI review found 1 potential issue(s):
- [High] The patch prevents an infinite recursion causing a kernel stack overflow by rejecting TCP_NODELAY in specific BPF_SOCK_OPS callbacks. However, it fails to address the exact same recursion vulnerability achievable via bpf_sk_setsockopt in a BPF TCP congestion control algorithm (BPF_PROG_TYPE_STRUCT_OPS).
--

commit 4ff4d98f6b4e3c3d90f9268761de84573f49fd86
Author: KaFai Wan <[email protected]>

bpf: Reject TCP_NODELAY in TCP header option callbacks

This patch rejects bpf_setsockopt(TCP_NODELAY) when called from BPF_SOCK_OPS_HDR_OPT_LEN_CB or BPF_SOCK_OPS_WRITE_HDR_OPT_CB to prevent unbounded recursion and kernel stack overflow.

net/core/filter.c BPF_CALL_5(bpf_sock_ops_setsockopt, struct bpf_sock_ops_kern *, bpf_sock, · line 5840

Quoted 13 lines
net/core/filter.c +0 -0
index fcfcb72663ca3..eae2f2c542716 100644
@@ -5833,6 +5833,12 @@ BPF_CALL_5(bpf_sock_ops_setsockopt, struct bpf_sock_ops_kern *, bpf_sock,
  	if (!is_locked_tcp_sock_ops(bpf_sock))
  		return -EOPNOTSUPP;

net/core/filter.c +5 -0
+	/* TCP_NODELAY triggers tcp_push_pending_frames() and re-enters these callbacks. */
+	if ((bpf_sock->op == BPF_SOCK_OPS_HDR_OPT_LEN_CB ||
+	     bpf_sock->op == BPF_SOCK_OPS_WRITE_HDR_OPT_CB) &&
+	    level == SOL_TCP && optname == TCP_NODELAY)
+		return -EOPNOTSUPP;

Does this same recursion vulnerability exist for BPF TCP congestion control algorithms using BPF_PROG_TYPE_STRUCT_OPS?

If a BPF congestion control algorithm invokes bpf_setsockopt(TCP_NODELAY) from its cwnd_event callback when handling CA_EVENT_TX_START, could it trigger the same unbounded recursion?

When the kernel transmits the first packet of a data train via tcp_transmit_skb(), it invokes tcp_event_data_sent(). Because tp->packets_out is not incremented until later, tcp_packets_in_flight(tp) evaluates to 0, triggering tcp_ca_event(sk, CA_EVENT_TX_START).

If the BPF program then calls bpf_setsockopt(TCP_NODELAY), it would result in this call chain:

tcp_transmit_skb()
  tcp_event_data_sent() -> invokes CA_EVENT_TX_START cwnd_event()
      bpf_setsockopt(TCP_NODELAY) tcp_push_pending_frames() tcp_write_xmit()

Quoted 4 lines

Since the outer tcp_transmit_skb() hasn't finished, the send head hasn't advanced. Wouldn't tcp_write_xmit() see the same SKB, attempt to transmit it again, and re-enter tcp_transmit_skb() causing an infinite recursion?

You are right. I can reproduce this.

Quoted 4 lines

Should the restriction on TCP_NODELAY be enforced at a broader level, such as inside _bpf_setsockopt(), to protect contexts holding the socket lock during TX paths?

We can check in sol_tcp_sockopt().

Quoted 5 lines
Quoted 4 lines

+
  return _bpf_setsockopt(bpf_sock->sk, level, optname, optval, optlen); }

Signature · 2 lines
Thanks,
KaFai
#12 lore raw

Re: [PATCH bpf v3 2/2] selftests/bpf: Test TCP_NODELAY in TCP hdr opt callbacks

In reply to KaFai Wan
Quoted 8 lines · KaFai Wan
Quoted 3 lines

tools/testing/selftests/bpf/prog_tests/tcp_hdr_options.c static void misc(void) · line 480

Quoted 1 line
tools/testing/selftests/bpf/prog_tests/tcp_hdr_options.c +1 -0
+	ret = setsockopt(sk_fds.active_fd, SOL_TCP, TCP_NODELAY, &true_val, sizeof(true_val));

Same comment as in v2. Why this setsockopt is needed?

Sorry I miss this. It's from the review of v1, my first version would break the syscall setsockopt and other CB besides HDR_OPT_LEN/WRITE_HDR_OPT. So in the test I check setsockopt() and bpf_setsockopt() in PASSIVE_ESTABLISHED_CB to make sure patch#1 would not break user space and other CB.

ic. Yep, remove it since v3 is not changing the syscall setsockopt.

Quoted 4 lines
Quoted 2 lines

The setsockopt in userspace is unnecessary.

Is bpf_setsockopt() in PASSIVE_ESTABLISHED_CB also unnecessary? I'll respin if they are unnecessary.

This one is fine. It checks if the bpf_setsockopt is not affectred in other CB.

#6 lore raw
In reply to KaFai Wan
Quoted 26 lines · KaFai Wan
Quoted 22 lines

Does this same recursion vulnerability exist for BPF TCP congestion control algorithms using BPF_PROG_TYPE_STRUCT_OPS?

If a BPF congestion control algorithm invokes bpf_setsockopt(TCP_NODELAY) from its cwnd_event callback when handling CA_EVENT_TX_START, could it trigger the same unbounded recursion?

When the kernel transmits the first packet of a data train via tcp_transmit_skb(), it invokes tcp_event_data_sent(). Because tp->packets_out is not incremented until later, tcp_packets_in_flight(tp) evaluates to 0, triggering tcp_ca_event(sk, CA_EVENT_TX_START).

If the BPF program then calls bpf_setsockopt(TCP_NODELAY), it would result in this call chain:

tcp_transmit_skb()
  tcp_event_data_sent() -> invokes CA_EVENT_TX_START cwnd_event()
      bpf_setsockopt(TCP_NODELAY) tcp_push_pending_frames() tcp_write_xmit()

Quoted 4 lines

Since the outer tcp_transmit_skb() hasn't finished, the send head hasn't advanced. Wouldn't tcp_write_xmit() see the same SKB, attempt to transmit it again, and re-enter tcp_transmit_skb() causing an infinite recursion?

Quoted 7 lines

You are right. I can reproduce this.

Quoted 4 lines

Should the restriction on TCP_NODELAY be enforced at a broader level, such as inside _bpf_setsockopt(), to protect contexts holding the socket lock during TX paths?

We can check in sol_tcp_sockopt().

I don't know how it can use the socket lock to single out this case. All bpf programs that are allowed to call bpf_setsockopt should have the sock lock held. Maybe I am missing something obvious.

In bpf_tcp_ca_get_func_proto, it checks what ops can do bpf_sk_setsockopt_proto. Right now, it rejects the "release" ops. One option is to create a new func_proto, bpf_sk_setsockopt_nodelay_proto, to reject TCP_NODELAY. Instead of checking cwnd_event[_tx_start] in bpf_tcp_ca_get_func_proto, I would return bpf_sk_setsockopt_nodelay_proto for all ops. We can revisit and be more selective in the future if the hammer turns out to be too big. "release" ops will remain disallowed from calling bpf_setsockopt.

author
#7 lore raw
Quoted 33 lines · Martin KaFai Lau
Quoted 32 lines · KaFai Wan
Quoted 22 lines

Does this same recursion vulnerability exist for BPF TCP congestion control algorithms using BPF_PROG_TYPE_STRUCT_OPS?

If a BPF congestion control algorithm invokes bpf_setsockopt(TCP_NODELAY) from its cwnd_event callback when handling CA_EVENT_TX_START, could it trigger the same unbounded recursion?

When the kernel transmits the first packet of a data train via tcp_transmit_skb(), it invokes tcp_event_data_sent(). Because tp->packets_out is not incremented until later, tcp_packets_in_flight(tp) evaluates to 0, triggering tcp_ca_event(sk, CA_EVENT_TX_START).

If the BPF program then calls bpf_setsockopt(TCP_NODELAY), it would result in this call chain:

tcp_transmit_skb()
  tcp_event_data_sent() -> invokes CA_EVENT_TX_START cwnd_event()
      bpf_setsockopt(TCP_NODELAY) tcp_push_pending_frames() tcp_write_xmit()

Quoted 4 lines

Since the outer tcp_transmit_skb() hasn't finished, the send head hasn't advanced. Wouldn't tcp_write_xmit() see the same SKB, attempt to transmit it again, and re-enter tcp_transmit_skb() causing an infinite recursion?

You are right. I can reproduce this.

Quoted 4 lines

Should the restriction on TCP_NODELAY be enforced at a broader level, such as inside _bpf_setsockopt(), to protect contexts holding the socket lock during TX paths?

Quoted 5 lines
Quoted 1 line

We can check in sol_tcp_sockopt().

I don't know how it can use the socket lock to single out this case. All bpf programs that are allowed to call bpf_setsockopt should have the sock lock held. Maybe I am missing something obvious.

I tried to find a way to determine if the sk is in tx state in tcp_transmit_skb(), but didn't succeed.

Quoted 8 lines

In bpf_tcp_ca_get_func_proto, it checks what ops can do bpf_sk_setsockopt_proto. Right now, it rejects the "release" ops. One option is to create a new func_proto, bpf_sk_setsockopt_nodelay_proto, to reject TCP_NODELAY. Instead of checking cwnd_event[_tx_start] in bpf_tcp_ca_get_func_proto, I would return bpf_sk_setsockopt_nodelay_proto for all ops. We can revisit and be more selective in the future if the hammer turns out to be too big. "release" ops will remain disallowed from calling bpf_setsockopt.

Great, I'll try this one. bpf_getsockopt(TCP_NODELAY) will not trigger infinite recursion, I will keep it as is.

Signature · 2 lines
Thanks,
KaFai