bpf: Reject TCP_NODELAY in TCP header option callbacks
Subsystems (6)
v3 cover bpf: Reject TCP_NODELAY in TCP header option callbacks
This small patchset is about avoid infinite recursion in TCP header option callbacks via TCP_NODELAY setsockopt.
v3:
- Remove CONFIG_INET check and add comment (Martin and Jiayuan)
- Fix the test case (Martin)
v2:
- Reject TCP_NODELAY in bpf_sock_ops_setsockopt() (AI and Martin)
https://lore.kernel.org/bpf/[email protected]/
v1:
https://lore.kernel.org/bpf/[email protected]/
---
KaFai Wan (2):
bpf: Reject TCP_NODELAY in TCP header option callbacks selftests/bpf: Test TCP_NODELAY in TCP hdr opt callbacks
net/core/filter.c | 6 ++++++ .../selftests/bpf/prog_tests/tcp_hdr_options.c | 12 +++++++++++- .../bpf/progs/test_misc_tcp_hdr_options.c | 15 ++++++++++++++- 3 files changed, 31 insertions(+), 2 deletions(-)
Signature · 1 line
2.43.0
PATCH v3 1/2 bpf: Reject TCP_NODELAY in TCP header option callbacks
A BPF_SOCK_OPS program can enable BPF_SOCK_OPS_WRITE_HDR_OPT_CB_FLAG and then call bpf_setsockopt(TCP_NODELAY) from BPF_SOCK_OPS_HDR_OPT_LEN_CB or BPF_SOCK_OPS_WRITE_HDR_OPT_CB.
In these callbacks, bpf_setsockopt(TCP_NODELAY) can reach __tcp_sock_set_nodelay(), which can call tcp_push_pending_frames().
From BPF_SOCK_OPS_HDR_OPT_LEN_CB, tcp_push_pending_frames() can call tcp_current_mss(), which calls tcp_established_options() and re-enters bpf_skops_hdr_opt_len().
BPF_SOCK_OPS_HDR_OPT_LEN_CB
-> bpf_setsockopt(TCP_NODELAY)
-> tcp_push_pending_frames()
-> tcp_current_mss()
-> tcp_established_options()
-> bpf_skops_hdr_opt_len()
-> BPF_SOCK_OPS_HDR_OPT_LEN_CBFrom BPF_SOCK_OPS_WRITE_HDR_OPT_CB, tcp_push_pending_frames() can call tcp_write_xmit(), which calls tcp_transmit_skb(). That path recomputes header option length through tcp_established_options() and bpf_skops_hdr_opt_len() before re-entering bpf_skops_write_hdr_opt().
BPF_SOCK_OPS_WRITE_HDR_OPT_CB
-> bpf_setsockopt(TCP_NODELAY)
-> tcp_push_pending_frames()
-> tcp_write_xmit()
-> tcp_transmit_skb()
-> tcp_established_options()
-> bpf_skops_hdr_opt_len()
-> bpf_skops_write_hdr_opt()
-> BPF_SOCK_OPS_WRITE_HDR_OPT_CBThis leads to unbounded recursion and can overflow the kernel stack.
Reject TCP_NODELAY with -EOPNOTSUPP in bpf_sock_ops_setsockopt() when bpf_setsockopt() is called from BPF_SOCK_OPS_HDR_OPT_LEN_CB or BPF_SOCK_OPS_WRITE_HDR_OPT_CB.
--- net/core/filter.c | 6 ++++++ 1 file changed, 6 insertions(+)
net/core/filter.c +6 -0
@@ -5833,6 +5833,12 @@ BPF_CALL_5(bpf_sock_ops_setsockopt, struct bpf_sock_ops_kern *, bpf_sock,if (!is_locked_tcp_sock_ops (bpf_sock))return -EOPNOTSUPP ; +/* TCP_NODELAY triggers tcp_push_pending_frames() and re-enters these callbacks. */ +if ((bpf_sock->op ==BPF_SOCK_OPS_HDR_OPT_LEN_CB || + bpf_sock->op ==BPF_SOCK_OPS_WRITE_HDR_OPT_CB ) && + level ==SOL_TCP && optname ==TCP_NODELAY ) +return -EOPNOTSUPP ; +return _bpf_setsockopt (bpf_sock->sk, level, optname, optval, optlen); }
Signature · 1 line
2.43.0
PATCH v3 2/2 selftests/bpf: Test TCP_NODELAY in TCP hdr opt callbacks
In reply to KaFai WanAdd a sockops selftest for the TCP_NODELAY restriction in BPF_SOCK_OPS_HDR_OPT_LEN_CB and BPF_SOCK_OPS_WRITE_HDR_OPT_CB.
With BPF_SOCK_OPS_WRITE_HDR_OPT_CB_FLAG enabled, bpf_setsockopt(TCP_NODELAY) returns -EOPNOTSUPP from BPF_SOCK_OPS_HDR_OPT_LEN_CB and BPF_SOCK_OPS_WRITE_HDR_OPT_CB, avoiding unbounded recursion and kernel stack overflow.
Other cases continue to work as before, including BPF_SOCK_OPS_PASSIVE_ESTABLISHED_CB and user space setsockopt(TCP_NODELAY).
--- .../selftests/bpf/prog_tests/tcp_hdr_options.c | 12 +++++++++++- .../bpf/progs/test_misc_tcp_hdr_options.c | 15 ++++++++++++++- 2 files changed, 25 insertions(+), 2 deletions(-)
tools/testing/selftests/bpf/prog_tests/tcp_hdr_options.c +11 -1
@@ -461,7 +461,7 @@ static void misc(void)const unsigned int nr_data =2 ;struct bpf_link *link;struct sk_fds sk_fds; -int i, ret; +int i, ret, true_val =1 ; lport_linum_map_fd =bpf_map__fd (misc_skel->maps.lport_linum_map);
@@ -477,6 +477,10 @@ static void misc(void)return ; } + ret =setsockopt (sk_fds.active_fd,SOL_TCP ,TCP_NODELAY , &true_val,sizeof (true_val)); +if (!ASSERT_OK (ret,"setsockopt(TCP_NODELAY)" )) + goto check_linum; +for (i =0 ; i < nr_data; i++) {/* MSG_EOR to ensure skb will not be combined */ ret = send(sk_fds.active_fd, send_msg, sizeof(send_msg),
@@ -507,6 +511,12 @@ static void misc(void)ASSERT_EQ (misc_skel->bss->nr_hwtstamp,0 ,"nr_hwtstamp" ); +ASSERT_TRUE (misc_skel->data->nodelay_est_ok,"unexpected nodelay_est_ok" ); + +ASSERT_TRUE (misc_skel->data->nodelay_hdr_len_err,"unexpected nodelay_hdr_len_err" ); + +ASSERT_TRUE (misc_skel->data->nodelay_write_hdr_err,"unexpected nodelay_write_hdr_err" ); + check_linum:ASSERT_FALSE (check_error_linum (&sk_fds),"check_error_linum" );sk_fds_close (&sk_fds);
tools/testing/selftests/bpf/progs/test_misc_tcp_hdr_options.c +14 -1
@@ -29,6 +29,10 @@ unsigned int nr_syn = 0;unsigned int nr_fin =0 ;unsigned int nr_hwtstamp =0 ; +bool nodelay_est_ok = true; +bool nodelay_hdr_len_err = true; +bool nodelay_write_hdr_err = true; +/* Check the header received from the active side */ static int __check_active_hdr_in (struct bpf_sock_ops *skops,bool check_syn) {
@@ -300,7 +304,7 @@ static int handle_passive_estab(struct bpf_sock_ops *skops) SEC("sockops")int misc_estab (struct bpf_sock_ops *skops) { -int true_val =1 ; +int true_val =1 , false_val =0 , ret;switch (skops->op) {case BPF_SOCK_OPS_TCP_LISTEN_CB :
@@ -316,10 +320,19 @@ int misc_estab(struct bpf_sock_ops *skops)case BPF_SOCK_OPS_PARSE_HDR_OPT_CB :return handle_parse_hdr (skops);case BPF_SOCK_OPS_HDR_OPT_LEN_CB : + ret =bpf_setsockopt (skops,SOL_TCP ,TCP_NODELAY , &true_val,sizeof (true_val)); + nodelay_hdr_len_err &= ret == -EOPNOTSUPP ; +return handle_hdr_opt_len (skops);case BPF_SOCK_OPS_WRITE_HDR_OPT_CB : + ret =bpf_setsockopt (skops,SOL_TCP ,TCP_NODELAY , &true_val,sizeof (true_val)); + nodelay_write_hdr_err &= ret == -EOPNOTSUPP ; +return handle_write_hdr_opt (skops);case BPF_SOCK_OPS_PASSIVE_ESTABLISHED_CB : + ret =bpf_setsockopt (skops,SOL_TCP ,TCP_NODELAY , &false_val,sizeof (false_val)); + nodelay_est_ok &= ret ==0 ; +return handle_passive_estab (skops); }
Signature · 1 line
2.43.0
[email protected]
AI reviewDoes this fix also need to block TCP_CORK? When bpf_setsockopt() is called with SOL_TCP and TCP_CORK where the value is 0 (disabling cork), the call path goes through:
Quoted 5 lines
net/core/filter.c +0
-0
net/core/filter.c BPF_CALL_5(bpf_sock_ops_setsockopt, struct bpf_sock_ops_kern *, bpf_sock, · line 5841
Quoted 9 lines
net/core/filter.c +6
-0@@ -5833,6 +5833,12 @@ BPF_CALL_5(bpf_sock_ops_setsockopt, struct bpf_sock_ops_kern *, bpf_sock,if (!is_locked_tcp_sock_ops (bpf_sock))return -EOPNOTSUPP ; +/* TCP_NODELAY triggers tcp_push_pending_frames() and re-enters these callbacks. */ +if ((bpf_sock->op ==BPF_SOCK_OPS_HDR_OPT_LEN_CB || + bpf_sock->op ==BPF_SOCK_OPS_WRITE_HDR_OPT_CB ) && + level ==SOL_TCP && optname ==TCP_NODELAY ) +return -EOPNOTSUPP ; +
Does this fix also need to block TCP_CORK? When bpf_setsockopt() is called with SOL_TCP and TCP_CORK where the value is 0 (disabling cork), the call path goes through:
bpf_sock_ops_setsockopt()
-> _bpf_setsockopt()
-> __bpf_setsockopt()
-> sol_tcp_sockopt()
-> do_tcp_setsockopt()
-> __tcp_sock_set_cork(sk, false)In __tcp_sock_set_cork() at net/ipv4/tcp.c, when cork is disabled, it unconditionally calls tcp_push_pending_frames(sk):
net/ipv4/tcp.c:__tcp_sock_set_cork() {
...
tcp->nonagle |= TCP_NAGLE_OFF|TCP_NAGLE_PUSH;
tcp_push_pending_frames(sk);
}This triggers the same recursion chain described in the commit message:
tcp_push_pending_frames()
-> tcp_current_mss()
-> tcp_established_options()
-> bpf_skops_hdr_opt_len()
-> BPF_CGROUP_RUN_PROG_SOCK_OPS_SK (re-enters with
op=BPF_SOCK_OPS_HDR_OPT_LEN_CB)causing unbounded recursion and kernel stack overflow. Note that enabling TCP_CORK (val=1) is safe since it doesn't call tcp_push_pending_frames(), only disabling it (val=0) does.
net/core/filter.c BPF_CALL_5(bpf_sock_ops_setsockopt, struct bpf_sock_ops_kern *, bpf_sock, · line 5843
Quoted 2 lines
net/core/filter.c +0
-0return _bpf_setsockopt (bpf_sock->sk, level, optname, optval, optlen); }
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/24558395229
Quoted 41 lines · KaFai Wan
A
BPF_SOCK_OPSprogram can enableBPF_SOCK_OPS_WRITE_HDR_OPT_CB_FLAGand then callbpf_setsockopt(TCP_NODELAY)fromBPF_SOCK_OPS_HDR_OPT_LEN_CBorBPF_SOCK_OPS_WRITE_HDR_OPT_CB.In these callbacks,
bpf_setsockopt(TCP_NODELAY)can reach__tcp_sock_set_nodelay(), which can calltcp_push_pending_frames().From
BPF_SOCK_OPS_HDR_OPT_LEN_CB,tcp_push_pending_frames()can calltcp_current_mss(), which callstcp_established_options()and re-entersbpf_skops_hdr_opt_len().
BPF_SOCK_OPS_HDR_OPT_LEN_CB
->bpf_setsockopt(TCP_NODELAY)-> tcp_push_pending_frames() -> tcp_current_mss() -> tcp_established_options() -> bpf_skops_hdr_opt_len() -> BPF_SOCK_OPS_HDR_OPT_LEN_CBFrom
BPF_SOCK_OPS_WRITE_HDR_OPT_CB,tcp_push_pending_frames()can calltcp_write_xmit(), which callstcp_transmit_skb(). That path recomputes header option length throughtcp_established_options()andbpf_skops_hdr_opt_len()before re-enteringbpf_skops_write_hdr_opt().
BPF_SOCK_OPS_WRITE_HDR_OPT_CB
->bpf_setsockopt(TCP_NODELAY)-> tcp_push_pending_frames() -> tcp_write_xmit() -> tcp_transmit_skb() -> tcp_established_options() -> bpf_skops_hdr_opt_len() -> bpf_skops_write_hdr_opt() -> BPF_SOCK_OPS_WRITE_HDR_OPT_CBThis leads to unbounded recursion and can overflow the kernel stack.
Reject
TCP_NODELAYwith-EOPNOTSUPPinbpf_sock_ops_setsockopt()whenbpf_setsockopt()is called fromBPF_SOCK_OPS_HDR_OPT_LEN_CBorBPF_SOCK_OPS_WRITE_HDR_OPT_CB.
Quoted 6 lines
Reported-by Quan Sun <[email protected]>Reported-by Yinhao Hu <[email protected]>Reported-by Kaiyan Mei <[email protected]>Fixes 7e41df5dbba2 ("bpf: Add a few optnames to bpf_setsockopt")Signed-off-by KaFai Wan <[email protected]>
I think the result of AI review is false-positve.
Re: [PATCH bpf v3 2/2] selftests/bpf: Test TCP_NODELAY in TCP hdr opt callbacks
In reply to KaFai WanQuoted 8 lines · KaFai Wan
Add a sockops selftest for the
TCP_NODELAYrestriction inBPF_SOCK_OPS_HDR_OPT_LEN_CBandBPF_SOCK_OPS_WRITE_HDR_OPT_CB.With
BPF_SOCK_OPS_WRITE_HDR_OPT_CB_FLAGenabled,bpf_setsockopt(TCP_NODELAY)returns-EOPNOTSUPPfromBPF_SOCK_OPS_HDR_OPT_LEN_CBandBPF_SOCK_OPS_WRITE_HDR_OPT_CB, avoiding unbounded recursion and kernel stack overflow.
Quoted 5 lines
Other cases continue to work as before, including
BPF_SOCK_OPS_PASSIVE_ESTABLISHED_CBand user spacesetsockopt(TCP_NODELAY).Signed-off-by KaFai Wan <[email protected]>
A little nit below, no need to resend.
Quoted 10 lines
--- .../selftests/bpf/prog_tests/tcp_hdr_options.c | 12 +++++++++++- .../bpf/progs/test_misc_tcp_hdr_options.c | 15 ++++++++++++++- 2 files changed, 25 insertions(+), 2 deletions(-)tools/testing/selftests/bpf/prog_tests/tcp_hdr_options.c +0
-0
tools/testing/selftests/bpf/prog_tests/tcp_hdr_options.c static void misc(void) · line 465
Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/tcp_hdr_options.c +1
-1@@ -461,7 +461,7 @@ static void misc(void)const unsigned int nr_data =2 ;struct bpf_link *link;struct sk_fds sk_fds; -int i, ret; +int i, ret, true_val =1 ;
NIT: please follow the reverse xmas tree variable ordering
Quoted 14 lines
tools/testing/selftests/bpf/prog_tests/tcp_hdr_options.c +4
-0lport_linum_map_fd =bpf_map__fd (misc_skel->maps.lport_linum_map);@@ -477,6 +477,10 @@ static void misc(void)return ; } + ret =setsockopt (sk_fds.active_fd,SOL_TCP ,TCP_NODELAY , &true_val,sizeof (true_val)); +if (!ASSERT_OK (ret,"setsockopt(TCP_NODELAY)" )) + goto check_linum; +for (i =0 ; i < nr_data; i++) {/* MSG_EOR to ensure skb will not be combined */ ret = send(sk_fds.active_fd, send_msg, sizeof(send_msg),
tools/testing/selftests/bpf/prog_tests/tcp_hdr_options.c static void misc(void) · line 519
Quoted 9 lines
tools/testing/selftests/bpf/prog_tests/tcp_hdr_options.c +6
-0@@ -507,6 +511,12 @@ static void misc(void)ASSERT_EQ (misc_skel->bss->nr_hwtstamp,0 ,"nr_hwtstamp" ); +ASSERT_TRUE (misc_skel->data->nodelay_est_ok,"unexpected nodelay_est_ok" ); + +ASSERT_TRUE (misc_skel->data->nodelay_hdr_len_err,"unexpected nodelay_hdr_len_err" ); + +ASSERT_TRUE (misc_skel->data->nodelay_write_hdr_err,"unexpected nodelay_write_hdr_err" ); +
NIT: It's would be misleading if you run ./test_progs with "-v" misc:PASS:unexpected nodelay_est_ok 0 nsec
"PASS:unexpected" ?
Quoted 8 lines
tools/testing/selftests/bpf/prog_tests/tcp_hdr_options.c +0
-0check_linum:ASSERT_FALSE (check_error_linum (&sk_fds),"check_error_linum" );sk_fds_close (&sk_fds);tools/testing/selftests/bpf/progs/test_misc_tcp_hdr_options.c +0
-0
tools/testing/selftests/bpf/progs/test_misc_tcp_hdr_options.c unsigned int nr_syn = 0; · line 34
Quoted 6 lines
tools/testing/selftests/bpf/progs/test_misc_tcp_hdr_options.c +3
-0@@ -29,6 +29,10 @@ unsigned int nr_syn = 0;unsigned int nr_fin =0 ;unsigned int nr_hwtstamp =0 ; +bool nodelay_est_ok = true; +bool nodelay_hdr_len_err = true; +bool nodelay_write_hdr_err = true;
I prefer "nodelay_hdr_len_reject"
Martin KaFai Lau
reviewerRe: [PATCH bpf v3 2/2] selftests/bpf: Test TCP_NODELAY in TCP hdr opt callbacks
In reply to KaFai WanQuoted 14 lines · KaFai Wan
tools/testing/selftests/bpf/prog_tests/tcp_hdr_options.c +1
-1@@ -461,7 +461,7 @@ static void misc(void)const unsigned int nr_data =2 ;struct bpf_link *link;struct sk_fds sk_fds; -int i, ret; +int i, ret, true_val =1 ; lport_linum_map_fd =bpf_map__fd (misc_skel->maps.lport_linum_map);
tools/testing/selftests/bpf/prog_tests/tcp_hdr_options.c static void misc(void) · line 480
Quoted 4 lines
tools/testing/selftests/bpf/prog_tests/tcp_hdr_options.c +1
-0@@ -477,6 +477,10 @@ static void misc(void)return ; } + ret =setsockopt (sk_fds.active_fd,SOL_TCP ,TCP_NODELAY , &true_val,sizeof (true_val));
Same comment as in v2. Why this setsockopt is needed? The setsockopt in userspace is unnecessary. In the future, we may need to understand why it is needed here in the first place.
Re: [PATCH bpf v3 2/2] selftests/bpf: Test TCP_NODELAY in TCP hdr opt callbacks
Quoted 14 lines · Martin KaFai Lau
Quoted 13 lines · KaFai Wan
tools/testing/selftests/bpf/prog_tests/tcp_hdr_options.c +1
-1@@ -461,7 +461,7 @@ static void misc(void) const unsigned int nr_data = 2; struct bpf_link *link; struct sk_fds sk_fds; -int i, ret; +int i, ret, true_val =1 ;
lport_linum_map_fd=bpf_map__fd(misc_skel->maps.lport_linum_map);
Quoted 8 lines
tools/testing/selftests/bpf/prog_tests/tcp_hdr_options.c static void misc(void) · line 480
Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/tcp_hdr_options.c +0
-0@@ -477,6 +477,10 @@ static void misc(void) return; }tools/testing/selftests/bpf/prog_tests/tcp_hdr_options.c +1
-0+ ret =setsockopt (sk_fds.active_fd,SOL_TCP ,TCP_NODELAY , &true_val,sizeof (true_val));Same comment as in v2. Why this setsockopt is needed?
Sorry I miss this. It's from the review of v1, my first version would break the syscall setsockopt and other CB besides HDR_OPT_LEN/WRITE_HDR_OPT. So in the test I check setsockopt() and bpf_setsockopt() in PASSIVE_ESTABLISHED_CB to make sure patch#1 would not break user space and other CB.
Quoted 1 line
The setsockopt in userspace is unnecessary.
Is bpf_setsockopt() in PASSIVE_ESTABLISHED_CB also unnecessary? I'll respin if they are unnecessary.
Quoted 2 lines
In the future,
we may need to understand why it is needed here in the first place.
Okay, I'll remember that. Thanks for the review and guidance.
Signature · 2 lines
Thanks, KaFai
[email protected]
AI reviewSashiko AI review found 1 potential issue(s): - [High] The patch prevents an infinite recursion causing a kernel stack overflow by rejecting TCP_NODELAY in specific BPF_SOCK_OPS callbacks. However…
Sashiko AI review found 1 potential issue(s):
- [High] The patch prevents an infinite recursion causing a kernel stack overflow by rejecting TCP_NODELAY in specific BPF_SOCK_OPS callbacks. However, it fails to address the exact same recursion vulnerability achievable via bpf_sk_setsockopt in a BPF TCP congestion control algorithm (BPF_PROG_TYPE_STRUCT_OPS). --
commit 4ff4d98f6b4e3c3d90f9268761de84573f49fd86
Author: KaFai Wan <[email protected]>
bpf: Reject TCP_NODELAY in TCP header option callbacks
This patch rejects bpf_setsockopt(TCP_NODELAY) when called from BPF_SOCK_OPS_HDR_OPT_LEN_CB or BPF_SOCK_OPS_WRITE_HDR_OPT_CB to prevent unbounded recursion and kernel stack overflow.
net/core/filter.c BPF_CALL_5(bpf_sock_ops_setsockopt, struct bpf_sock_ops_kern *, bpf_sock, · line 5840
Quoted 13 lines
net/core/filter.c +5
-0@@ -5833,6 +5833,12 @@ BPF_CALL_5(bpf_sock_ops_setsockopt, struct bpf_sock_ops_kern *, bpf_sock,if (!is_locked_tcp_sock_ops (bpf_sock))return -EOPNOTSUPP ; +/* TCP_NODELAY triggers tcp_push_pending_frames() and re-enters these callbacks. */ +if ((bpf_sock->op ==BPF_SOCK_OPS_HDR_OPT_LEN_CB || + bpf_sock->op ==BPF_SOCK_OPS_WRITE_HDR_OPT_CB ) && + level ==SOL_TCP && optname ==TCP_NODELAY ) +return -EOPNOTSUPP ;
Does this same recursion vulnerability exist for BPF TCP congestion control algorithms using BPF_PROG_TYPE_STRUCT_OPS?
If a BPF congestion control algorithm invokes bpf_setsockopt(TCP_NODELAY) from its cwnd_event callback when handling CA_EVENT_TX_START, could it trigger the same unbounded recursion?
When the kernel transmits the first packet of a data train via tcp_transmit_skb(), it invokes tcp_event_data_sent(). Because tp->packets_out is not incremented until later, tcp_packets_in_flight(tp) evaluates to 0, triggering tcp_ca_event(sk, CA_EVENT_TX_START).
If the BPF program then calls bpf_setsockopt(TCP_NODELAY), it would result in this call chain:
tcp_transmit_skb()
tcp_event_data_sent() -> invokes CA_EVENT_TX_START
cwnd_event()
bpf_setsockopt(TCP_NODELAY)
tcp_push_pending_frames()
tcp_write_xmit()
Since the outer tcp_transmit_skb() hasn't finished, the send head hasn't advanced. Wouldn't tcp_write_xmit() see the same SKB, attempt to transmit it again, and re-enter tcp_transmit_skb() causing an infinite recursion?
Should the restriction on TCP_NODELAY be enforced at a broader level, such as inside _bpf_setsockopt(), to protect contexts holding the socket lock during TX paths?
Quoted 4 lines
net/core/filter.c +1
-0+return _bpf_setsockopt (bpf_sock->sk, level, optname, optval, optlen); }
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Quoted 54 lines · On Sat, 2026-04-18 at 09:22 +0000, [email protected] wrote:
Sashiko AI review found 1 potential issue(s):
- [High] The patch prevents an infinite recursion causing a kernel stack overflow by rejectingTCP_NODELAYin specificBPF_SOCK_OPScallbacks. However, it fails to address the exact same recursion vulnerability achievable viabpf_sk_setsockoptin a BPF TCP congestion control algorithm (BPF_PROG_TYPE_STRUCT_OPS).
--commit
4ff4d98f6b4e3c3d90f9268761de84573f49fd86
Author: KaFai Wan <[email protected]>bpf: Reject
TCP_NODELAYin TCP header option callbacksThis patch rejects
bpf_setsockopt(TCP_NODELAY)when called fromBPF_SOCK_OPS_HDR_OPT_LEN_CBorBPF_SOCK_OPS_WRITE_HDR_OPT_CBto prevent unbounded recursion and kernel stack overflow.net/core/filter.c BPF_CALL_5(bpf_sock_ops_setsockopt, struct bpf_sock_ops_kern *, bpf_sock, · line 5840
Quoted 13 lines
net/core/filter.c +0
-0@@ -5833,6 +5833,12 @@ BPF_CALL_5(bpf_sock_ops_setsockopt, struct bpf_sock_ops_kern *, bpf_sock, if (!is_locked_tcp_sock_ops(bpf_sock)) return -EOPNOTSUPP;net/core/filter.c +5
-0+/* TCP_NODELAY triggers tcp_push_pending_frames() and re-enters these callbacks. */ +if ((bpf_sock->op ==BPF_SOCK_OPS_HDR_OPT_LEN_CB || + bpf_sock->op == BPF_SOCK_OPS_WRITE_HDR_OPT_CB) && + level == SOL_TCP && optname == TCP_NODELAY) +return -EOPNOTSUPP ;Does this same recursion vulnerability exist for BPF TCP congestion control algorithms using
BPF_PROG_TYPE_STRUCT_OPS?If a BPF congestion control algorithm invokes
bpf_setsockopt(TCP_NODELAY)from itscwnd_eventcallback when handlingCA_EVENT_TX_START, could it trigger the same unbounded recursion?When the kernel transmits the first packet of a data train via
tcp_transmit_skb(), it invokestcp_event_data_sent(). Becausetp->packets_outis not incremented until later,tcp_packets_in_flight(tp)evaluates to 0, triggeringtcp_ca_event(sk, CA_EVENT_TX_START).If the BPF program then calls
bpf_setsockopt(TCP_NODELAY), it would result in this call chain:
tcp_transmit_skb()
tcp_event_data_sent()-> invokesCA_EVENT_TX_STARTcwnd_event()
bpf_setsockopt(TCP_NODELAY)tcp_push_pending_frames()tcp_write_xmit()
Quoted 4 lines
Since the outer
tcp_transmit_skb()hasn't finished, the send head hasn't advanced. Wouldn'ttcp_write_xmit()see the same SKB, attempt to transmit it again, and re-entertcp_transmit_skb()causing an infinite recursion?
You are right. I can reproduce this.
Quoted 4 lines
Should the restriction on
TCP_NODELAYbe enforced at a broader level, such as inside_bpf_setsockopt(), to protect contexts holding the socket lock during TX paths?
We can check in sol_tcp_sockopt().
Quoted 5 lines
Quoted 4 lines
+
return_bpf_setsockopt(bpf_sock->sk, level, optname, optval, optlen); }
Signature · 2 lines
Thanks, KaFai
Martin KaFai Lau
reviewerRe: [PATCH bpf v3 2/2] selftests/bpf: Test TCP_NODELAY in TCP hdr opt callbacks
In reply to KaFai WanQuoted 8 lines · KaFai Wan
Quoted 3 lines
tools/testing/selftests/bpf/prog_tests/tcp_hdr_options.c static void misc(void) · line 480
Quoted 1 line
tools/testing/selftests/bpf/prog_tests/tcp_hdr_options.c +1
-0+ ret =setsockopt (sk_fds.active_fd,SOL_TCP ,TCP_NODELAY , &true_val,sizeof (true_val));Same comment as in v2. Why this setsockopt is needed?
Sorry I miss this. It's from the review of v1, my first version would break the syscall setsockopt and other CB besides HDR_OPT_LEN/WRITE_HDR_OPT. So in the test I check
setsockopt()andbpf_setsockopt()inPASSIVE_ESTABLISHED_CBto make sure patch#1 would not break user space and other CB.
ic. Yep, remove it since v3 is not changing the syscall setsockopt.
Quoted 4 lines
Quoted 2 lines
The setsockopt in userspace is unnecessary.
Is
bpf_setsockopt()inPASSIVE_ESTABLISHED_CBalso unnecessary? I'll respin if they are unnecessary.
This one is fine. It checks if the bpf_setsockopt is not affectred in other CB.
Martin KaFai Lau
reviewerQuoted 26 lines · KaFai Wan
Quoted 22 lines
Does this same recursion vulnerability exist for BPF TCP congestion control algorithms using
BPF_PROG_TYPE_STRUCT_OPS?If a BPF congestion control algorithm invokes
bpf_setsockopt(TCP_NODELAY)from itscwnd_eventcallback when handlingCA_EVENT_TX_START, could it trigger the same unbounded recursion?When the kernel transmits the first packet of a data train via
tcp_transmit_skb(), it invokestcp_event_data_sent(). Becausetp->packets_outis not incremented until later,tcp_packets_in_flight(tp)evaluates to 0, triggeringtcp_ca_event(sk, CA_EVENT_TX_START).If the BPF program then calls
bpf_setsockopt(TCP_NODELAY), it would result in this call chain:
tcp_transmit_skb()
tcp_event_data_sent()-> invokesCA_EVENT_TX_STARTcwnd_event()
bpf_setsockopt(TCP_NODELAY)tcp_push_pending_frames()tcp_write_xmit()Quoted 4 lines
Since the outer
tcp_transmit_skb()hasn't finished, the send head hasn't advanced. Wouldn'ttcp_write_xmit()see the same SKB, attempt to transmit it again, and re-entertcp_transmit_skb()causing an infinite recursion?
Quoted 7 lines
You are right. I can reproduce this.
Quoted 4 lines
Should the restriction on
TCP_NODELAYbe enforced at a broader level, such as inside_bpf_setsockopt(), to protect contexts holding the socket lock during TX paths?We can check in
sol_tcp_sockopt().
I don't know how it can use the socket lock to single out this case. All bpf programs that are allowed to call bpf_setsockopt should have the sock lock held. Maybe I am missing something obvious.
In bpf_tcp_ca_get_func_proto, it checks what ops can do bpf_sk_setsockopt_proto. Right now, it rejects the "release" ops. One option is to create a new func_proto, bpf_sk_setsockopt_nodelay_proto, to reject TCP_NODELAY. Instead of checking cwnd_event[_tx_start] in bpf_tcp_ca_get_func_proto, I would return bpf_sk_setsockopt_nodelay_proto for all ops. We can revisit and be more selective in the future if the hammer turns out to be too big. "release" ops will remain disallowed from calling bpf_setsockopt.
Quoted 33 lines · Martin KaFai Lau
Quoted 32 lines · KaFai Wan
Quoted 22 lines
Does this same recursion vulnerability exist for BPF TCP congestion control algorithms using
BPF_PROG_TYPE_STRUCT_OPS?If a BPF congestion control algorithm invokes
bpf_setsockopt(TCP_NODELAY)from itscwnd_eventcallback when handlingCA_EVENT_TX_START, could it trigger the same unbounded recursion?When the kernel transmits the first packet of a data train via
tcp_transmit_skb(), it invokestcp_event_data_sent(). Becausetp->packets_outis not incremented until later,tcp_packets_in_flight(tp)evaluates to 0, triggeringtcp_ca_event(sk, CA_EVENT_TX_START).If the BPF program then calls
bpf_setsockopt(TCP_NODELAY), it would result in this call chain:
tcp_transmit_skb()
tcp_event_data_sent()-> invokesCA_EVENT_TX_STARTcwnd_event()
bpf_setsockopt(TCP_NODELAY)tcp_push_pending_frames()tcp_write_xmit()Quoted 4 lines
Since the outer
tcp_transmit_skb()hasn't finished, the send head hasn't advanced. Wouldn'ttcp_write_xmit()see the same SKB, attempt to transmit it again, and re-entertcp_transmit_skb()causing an infinite recursion?You are right. I can reproduce this.
Quoted 4 lines
Should the restriction on
TCP_NODELAYbe enforced at a broader level, such as inside_bpf_setsockopt(), to protect contexts holding the socket lock during TX paths?
Quoted 5 lines
Quoted 1 line
We can check in
sol_tcp_sockopt().I don't know how it can use the socket lock to single out this case. All bpf programs that are allowed to call
bpf_setsockoptshould have the sock lock held. Maybe I am missing something obvious.
I tried to find a way to determine if the sk is in tx state in tcp_transmit_skb(), but didn't succeed.
Quoted 8 lines
In
bpf_tcp_ca_get_func_proto, it checks what ops can dobpf_sk_setsockopt_proto. Right now, it rejects the "release" ops. One option is to create a newfunc_proto,bpf_sk_setsockopt_nodelay_proto, to rejectTCP_NODELAY. Instead of checking cwnd_event[_tx_start] inbpf_tcp_ca_get_func_proto, I would returnbpf_sk_setsockopt_nodelay_protofor all ops. We can revisit and be more selective in the future if the hammer turns out to be too big. "release" ops will remain disallowed from callingbpf_setsockopt.
Great, I'll try this one. bpf_getsockopt(TCP_NODELAY) will not trigger infinite recursion, I will keep it as is.
Signature · 2 lines
Thanks, KaFai