net,bpf: fix null-ptr-deref in xdp_master_redirect() for bonding and add selftest
Subsystems (8)
v5 cover net,bpf: fix null-ptr-deref in xdp_master_redirect() for bonding and add selftest
syzkaller reported a kernel panic [1] with the following crash stack:
Call Trace: BUG: unable to handle page fault for address: ffff8ebd08580000 PF: supervisor write access in kernel mode PF: error_code(0x0002) - not-present page PGD 11f201067 P4D 11f201067 PUD 0 Oops: Oops: 0002 [#1] SMP PTI CPU: 2 UID: 0 PID: 451 Comm: test_progs Not tainted 6.19.0+ #161 PREEMPT_RT RIP: 0010:bond_rr_gen_slave_id+0x90/0xd0 RSP: 0018:ffffd3f4815f3448 EFLAGS: 00010246 RAX: 0000000000000001 RBX: 0000000000000001 RCX: ffff8ebc8728b17e RDX: 0000000000000000 RSI: ffffd3f4815f3538 RDI: ffff8ebc8abcce40 RBP: ffffd3f4815f3460 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000000 R12: ffffd3f4815f3538 R13: ffff8ebc8abcce40 R14: ffff8ebc8728b17f R15: ffff8ebc8728b170 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: ffff8ebd08580000 CR3: 000000010a808006 CR4: 0000000000770ef0 PKRU: 55555554 Call Trace: <TASK> bond_xdp_get_xmit_slave+0xc0/0x240 xdp_master_redirect+0x74/0xc0 bpf_prog_run_generic_xdp+0x2f2/0x3f0 do_xdp_generic+0x1fd/0x3d0 __netif_receive_skb_core.constprop.0+0x30d/0x1220 __netif_receive_skb_list_core+0xfc/0x250 netif_receive_skb_list_internal+0x20c/0x3d0 ? eth_type_trans+0x137/0x160 netif_receive_skb_list+0x25/0x140 xdp_test_run_batch.constprop.0+0x65b/0x6e0 bpf_test_run_xdp_live+0x1ec/0x3b0 bpf_prog_test_run_xdp+0x49d/0x6e0 __sys_bpf+0x446/0x27b0 __x64_sys_bpf+0x1a/0x30 x64_sys_call+0x146c/0x26e0 do_syscall_64+0xd3/0x1510 entry_SYSCALL_64_after_hwframe+0x76/0x7e
Problem Description
bond_rr_gen_slave_id() dereferences bond->rr_tx_counter without a NULL check. rr_tx_counter is a per-CPU counter only allocated in bond_open() when the bond mode is round-robin. If the bond device was never brought up, rr_tx_counter remains NULL.
The XDP redirect path can reach this code even when the bond is not up:bpf_master_redirect_enabled_key is a global static key, so when any bond device has native XDP attached, the XDP_TX -> xdp_master_redirect() interception is enabled for all bond slaves system-wide.
Solution
Patch 1: Add a NULL check with unlikely() in bond_rr_gen_slave_id() before dereferencing rr_tx_counter. When rr_tx_counter is NULL (bond was never opened), fall back to get_random_u32() for slave selection. The existing allocation in bond_open() is kept, with WRITE_ONCE() added to pair with the READ_ONCE() in the NULL check.
Patch 2: Add a selftest that reproduces the above scenario.
Changes since v4:
https://lore.kernel.org/netdev/[email protected]/
- Reverted unconditional alloc in bond_init(); instead add a NULL check with unlikely()/READ_ONCE() in bond_rr_gen_slave_id() and WRITE_ONCE() in bond_open(), avoiding memory waste for non-RR modes (Suggested by Nikolay Aleksandrov, patch by Jay Vosburgh)
Changes since v3:
https://lore.kernel.org/netdev/[email protected]/T/#t
- Added code comment and commit log explaining why rr_tx_counter is allocated unconditionally for all modes (Suggested by Jay Vosburgh)
Changes since v2:
https://lore.kernel.org/netdev/[email protected]/T/#t
- Moved allocation from bond_create_init() helper into bond_init() (ndo_init), which is the natural single point covering both creation paths and also handles post-creation mode changes to round-robin
Changes since v1:
https://lore.kernel.org/netdev/[email protected]/T/#t
- Moved the guard for NULL rr_tx_counter from xdp_master_redirect() into the bonding subsystem itself
(Suggested by Sebastian Andrzej Siewior <[email protected]>)
[1] https://syzkaller.appspot.com/bug?extid=80e046b8da2820b6ba73
Jiayuan Chen (2):
bonding: fix null-ptr-deref in bond_rr_gen_slave_id() selftests/bpf: add test for xdp_master_redirect with bond not up
drivers/net/bonding/bond_main.c | 9 +- .../selftests/bpf/prog_tests/xdp_bonding.c | 101 +++++++++++++++++- 2 files changed, 106 insertions(+), 4 deletions(-)
Signature · 1 line
2.43.0
PATCH v5 1/2 bonding: fix null-ptr-deref in bond_rr_gen_slave_id()
From: Jiayuan Chen <[email protected]>
bond_rr_gen_slave_id() dereferences bond->rr_tx_counter without a NULL check. rr_tx_counter is a per-CPU counter only allocated in bond_open() when the bond mode is round-robin. If the bond device was never brought up, rr_tx_counter remains NULL, causing a null-ptr-deref.
The XDP redirect path can reach this code even when the bond is not up:bpf_master_redirect_enabled_key is a global static key, so when any bond device has native XDP attached, the XDP_TX -> xdp_master_redirect() interception is enabled for all bond slaves system-wide. This allows the path xdp_master_redirect() -> bond_xdp_get_xmit_slave() -> bond_xdp_xmit_roundrobin_slave_get() -> bond_rr_gen_slave_id() to be reached on a bond that was never opened.
Fix this by adding a NULL check with unlikely() in bond_rr_gen_slave_id() before dereferencing rr_tx_counter. When rr_tx_counter is NULL (bond was never opened), fall back to get_random_u32() for slave selection. The allocation in bond_open() is kept, with WRITE_ONCE() added to safely publish the pointer to the XDP read side. A plain read suffices for the !bond->rr_tx_counter guard in bond_open() itself, as bond_open() runs under RTNL lock and is the only writer of rr_tx_counter.
--- drivers/net/bonding/bond_main.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
drivers/net/bonding/bond_main.c +7 -2
@@ -4290,9 +4290,11 @@ static int bond_open(struct net_device *bond_dev)struct slave *slave;if (BOND_MODE (bond) ==BOND_MODE_ROUNDROBIN && !bond->rr_tx_counter) { - bond->rr_tx_counter =alloc_percpu (u32); -if (!bond->rr_tx_counter) +u32 __percpu *rr_tx_tmp =alloc_percpu (u32); + +if (!rr_tx_tmp)return -ENOMEM ; +WRITE_ONCE (bond->rr_tx_counter, rr_tx_tmp); }/* reset slave->backup and slave->inactive */
@@ -4883,6 +4885,9 @@ static u32 bond_rr_gen_slave_id(struct bonding *bond)struct reciprocal_value reciprocal_packets_per_slave;int packets_per_slave = bond->params.packets_per_slave; +if (unlikely (!READ_ONCE (bond->rr_tx_counter))) +return get_random_u32 (); +switch (packets_per_slave) {case 0 : slave_id =get_random_u32 ();
Signature · 1 line
2.43.0
PATCH v5 2/2 selftests/bpf: add test for xdp_master_redirect with bond not up
In reply to Jiayuan ChenFrom: Jiayuan Chen <[email protected]>
Add a selftest that reproduces the null-ptr-deref in bond_rr_gen_slave_id() when XDP redirect targets a bond device in round-robin mode that was never brought up. The test verifies the fix by ensuring no crash occurs.
Test setup:
- bond0: active-backup mode, UP, with native XDP (enables bpf_master_redirect_enabled_key globally)
- bond1: round-robin mode, never UP
- veth1: slave of bond1, with generic XDP (XDP_TX)
- BPF_PROG_TEST_RUN with live frames triggers the redirect path
--- .../selftests/bpf/prog_tests/xdp_bonding.c | 101 +++++++++++++++++- 1 file changed, 99 insertions(+), 2 deletions(-)
tools/testing/selftests/bpf/prog_tests/xdp_bonding.c +99 -2
@@ -191,13 +191,18 @@ static int bonding_setup(struct skeletons *skeletons, int mode, int xmit_policy,return -1 ; } -static void bonding_cleanup (struct skeletons *skeletons) +static void link_cleanup (struct skeletons *skeletons) { -restore_root_netns ();while (skeletons->nlinks) { skeletons->nlinks--;bpf_link__destroy (skeletons->links[skeletons->nlinks]); } +} + +static void bonding_cleanup (struct skeletons *skeletons) +{ +restore_root_netns (); +link_cleanup (skeletons);ASSERT_OK (system ("ip link delete bond1" ),"delete bond1" );ASSERT_OK (system ("ip link delete veth1_1" ),"delete veth1_1" );ASSERT_OK (system ("ip link delete veth1_2" ),"delete veth1_2" );
@@ -493,6 +498,95 @@ static void test_xdp_bonding_nested(struct skeletons *skeletons)system ("ip link del bond_nest2" ); } +/* +* Test that XDP redirect via xdp_master_redirect() does not crash when +* the bond master device is not up. When bond is in round-robin mode but +* never opened, rr_tx_counter is NULL. +*/ +static void test_xdp_bonding_redirect_no_up (struct skeletons *skeletons) +{ +struct nstoken *nstoken =NULL ; +int xdp_pass_fd, xdp_tx_fd; +int veth1_ifindex; +int err; +char pkt[ETH_HLEN +1 ]; +struct xdp_md ctx_in = {}; + +DECLARE_LIBBPF_OPTS (bpf_test_run_opts, opts, + .data_in = &pkt, + .data_size_in = sizeof(pkt), + .ctx_in = &ctx_in, + .ctx_size_in = sizeof(ctx_in), + .flags = BPF_F_TEST_XDP_LIVE_FRAMES, + .repeat = 1, + .batch_size = 1, + ); + +/* We can't use bonding_setup() because bond will be active */ +SYS (out,"ip netns add ns_rr_no_up" ); + nstoken =open_netns ("ns_rr_no_up" ); +if (!ASSERT_OK_PTR (nstoken,"open ns_rr_no_up" )) + goto out; + +/* bond0: active-backup, UP with slave veth0. +* Attaching native XDP to bond0 enables bpf_master_redirect_enabled_key +* globally. +*/ +SYS (out,"ip link add bond0 type bond mode active-backup" ); +SYS (out,"ip link add veth0 type veth peer name veth0p" ); +SYS (out,"ip link set veth0 master bond0" ); +SYS (out,"ip link set bond0 up" ); +SYS (out,"ip link set veth0p up" ); + +/* bond1: round-robin, never UP -> rr_tx_counter stays NULL */ +SYS (out,"ip link add bond1 type bond mode balance-rr" ); +SYS (out,"ip link add veth1 type veth peer name veth1p" ); +SYS (out,"ip link set veth1 master bond1" ); + + veth1_ifindex =if_nametoindex ("veth1" ); +if (!ASSERT_GT (veth1_ifindex,0 ,"veth1_ifindex" )) + goto out; + +/* Attach native XDP to bond0 -> enables global redirect key */ +if (xdp_attach (skeletons, skeletons->xdp_tx->progs.xdp_tx,"bond0" )) + goto out; + +/* Attach generic XDP (XDP_TX) to veth1. +* When packets arrive at veth1 via netif_receive_skb, do_xdp_generic() +* runs this program. XDP_TX + bond slave triggers xdp_master_redirect(). +*/ + xdp_tx_fd =bpf_program__fd (skeletons->xdp_tx->progs.xdp_tx); +if (!ASSERT_GE (xdp_tx_fd,0 ,"xdp_tx prog_fd" )) + goto out; + + err =bpf_xdp_attach (veth1_ifindex, xdp_tx_fd, +XDP_FLAGS_SKB_MODE ,NULL ); +if (!ASSERT_OK (err,"attach generic XDP to veth1" )) + goto out; + +/* Run BPF_PROG_TEST_RUN with XDP_PASS live frames on veth1. +* XDP_PASS frames become SKBs with skb->dev = veth1, entering +* netif_receive_skb -> do_xdp_generic -> xdp_master_redirect. +* Without the fix, bond_rr_gen_slave_id() dereferences NULL +* rr_tx_counter and crashes. +*/ + xdp_pass_fd =bpf_program__fd (skeletons->xdp_dummy->progs.xdp_dummy_prog); +if (!ASSERT_GE (xdp_pass_fd,0 ,"xdp_pass prog_fd" )) + goto out; + +memset (pkt,0 ,sizeof (pkt)); + ctx_in.data_end =sizeof (pkt); + ctx_in.ingress_ifindex = veth1_ifindex; + + err =bpf_prog_test_run_opts (xdp_pass_fd, &opts); +ASSERT_OK (err,"xdp_pass test_run should not crash" ); + +out: +link_cleanup (skeletons); +close_netns (nstoken); +SYS_NOFAIL ("ip netns del ns_rr_no_up" ); +} +static void test_xdp_bonding_features (struct skeletons *skeletons) {LIBBPF_OPTS (bpf_xdp_query_opts, query_opts);
@@ -738,6 +832,9 @@ void serial_test_xdp_bonding(void)if (test__start_subtest ("xdp_bonding_redirect_multi" ))test_xdp_bonding_redirect_multi (&skeletons); +if (test__start_subtest ("xdp_bonding_redirect_no_up" )) +test_xdp_bonding_redirect_no_up (&skeletons); + out:xdp_dummy__destroy (skeletons.xdp_dummy);xdp_tx__destroy (skeletons.xdp_tx);
Signature · 1 line
2.43.0
Quoted 31 lines · Jiayuan Chen
syzkaller reported a kernel panic [1] with the following crash stack:
Call Trace: BUG: unable to handle page fault for address: ffff8ebd08580000 PF: supervisor write access in kernel mode PF: error_code(0x0002) - not-present page PGD 11f201067 P4D 11f201067 PUD 0 Oops: Oops: 0002 [#1] SMP PTI CPU: 2 UID: 0 PID: 451 Comm: test_progs Not tainted 6.19.0+ #161 PREEMPT_RT RIP: 0010:bond_rr_gen_slave_id+0x90/0xd0 RSP: 0018:ffffd3f4815f3448 EFLAGS: 00010246 RAX: 0000000000000001 RBX: 0000000000000001 RCX: ffff8ebc8728b17e RDX: 0000000000000000 RSI: ffffd3f4815f3538 RDI: ffff8ebc8abcce40 RBP: ffffd3f4815f3460 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000000 R12: ffffd3f4815f3538 R13: ffff8ebc8abcce40 R14: ffff8ebc8728b17f R15: ffff8ebc8728b170 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: ffff8ebd08580000 CR3: 000000010a808006 CR4: 0000000000770ef0 PKRU: 55555554 Call Trace: <TASK> bond_xdp_get_xmit_slave+0xc0/0x240 xdp_master_redirect+0x74/0xc0 bpf_prog_run_generic_xdp+0x2f2/0x3f0 do_xdp_generic+0x1fd/0x3d0 __netif_receive_skb_core.constprop.0+0x30d/0x1220 __netif_receive_skb_list_core+0xfc/0x250 netif_receive_skb_list_internal+0x20c/0x3d0 ? eth_type_trans+0x137/0x160 netif_receive_skb_list+0x25/0x140
Quoted 8 lines
xdp_test_run_batch.constprop.0+0x65b/0x6e0 bpf_test_run_xdp_live+0x1ec/0x3b0 bpf_prog_test_run_xdp+0x49d/0x6e0 __sys_bpf+0x446/0x27b0 __x64_sys_bpf+0x1a/0x30 x64_sys_call+0x146c/0x26e0 do_syscall_64+0xd3/0x1510 entry_SYSCALL_64_after_hwframe+0x76/0x7e
Please, can you always provide symbols in such traces ? You can use scripts/decode_stacktrace.sh to make the trace really nice, instead of ugly.
Quoted 59 lines
Problem Description
bond_rr_gen_slave_id()dereferencesbond->rr_tx_counterwithout a NULL check.rr_tx_counteris a per-CPU counter only allocated inbond_open()when the bond mode is round-robin. If the bond device was never brought up,rr_tx_counterremains NULL.The XDP redirect path can reach this code even when the bond is not up:
bpf_master_redirect_enabled_keyis a global static key, so when any bond device has native XDP attached, theXDP_TX->xdp_master_redirect()interception is enabled for all bond slaves system-wide.Solution
Patch 1: Add a NULL check with
unlikely()inbond_rr_gen_slave_id()before dereferencingrr_tx_counter. Whenrr_tx_counteris NULL (bond was never opened), fall back toget_random_u32()for slave selection. The existing allocation inbond_open()is kept, withWRITE_ONCE()added to pair with theREAD_ONCE()in the NULL check.
Patch 2: Add a selftest that reproduces the above scenario.Changes since v4:
https://lore.kernel.org/netdev/[email protected]/
- Reverted unconditional alloc inbond_init(); instead add a NULL check with unlikely()/READ_ONCE() inbond_rr_gen_slave_id()andWRITE_ONCE()inbond_open(), avoiding memory waste for non-RR modes (Suggested by Nikolay Aleksandrov, patch by Jay Vosburgh)Changes since v3:
https://lore.kernel.org/netdev/[email protected]/T/#t
- Added code comment and commit log explaining whyrr_tx_counteris allocated unconditionally for all modes (Suggested by Jay Vosburgh)Changes since v2:
https://lore.kernel.org/netdev/[email protected]/T/#t
- Moved allocation frombond_create_init()helper intobond_init()(ndo_init), which is the natural single point covering both creation paths and also handles post-creation mode changes to round-robinChanges since v1:
https://lore.kernel.org/netdev/[email protected]/T/#t
- Moved the guard for NULLrr_tx_counterfromxdp_master_redirect()into the bonding subsystem itself
(Suggested by Sebastian Andrzej Siewior <[email protected]>)[1] https://syzkaller.appspot.com/bug?extid=80e046b8da2820b6ba73
Jiayuan Chen (2):
bonding: fix null-ptr-deref inbond_rr_gen_slave_id()selftests/bpf: add test forxdp_master_redirectwith bond not updrivers/net/bonding/bond_main.c | 9 +- .../selftests/bpf/prog_tests/xdp_bonding.c | 101 +++++++++++++++++- 2 files changed, 106 insertions(+), 4 deletions(-)--
2.43.0
Quoted 43 lines · Eric Dumazet
Quoted 31 lines · Jiayuan Chen
syzkaller reported a kernel panic [1] with the following crash stack:
Call Trace: BUG: unable to handle page fault for address: ffff8ebd08580000 PF: supervisor write access in kernel mode PF: error_code(0x0002) - not-present page PGD 11f201067 P4D 11f201067 PUD 0 Oops: Oops: 0002 [#1] SMP PTI CPU: 2 UID: 0 PID: 451 Comm: test_progs Not tainted 6.19.0+ #161 PREEMPT_RT RIP: 0010:bond_rr_gen_slave_id+0x90/0xd0 RSP: 0018:ffffd3f4815f3448 EFLAGS: 00010246 RAX: 0000000000000001 RBX: 0000000000000001 RCX: ffff8ebc8728b17e RDX: 0000000000000000 RSI: ffffd3f4815f3538 RDI: ffff8ebc8abcce40 RBP: ffffd3f4815f3460 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000000 R12: ffffd3f4815f3538 R13: ffff8ebc8abcce40 R14: ffff8ebc8728b17f R15: ffff8ebc8728b170 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: ffff8ebd08580000 CR3: 000000010a808006 CR4: 0000000000770ef0 PKRU: 55555554 Call Trace: <TASK> bond_xdp_get_xmit_slave+0xc0/0x240 xdp_master_redirect+0x74/0xc0 bpf_prog_run_generic_xdp+0x2f2/0x3f0 do_xdp_generic+0x1fd/0x3d0 __netif_receive_skb_core.constprop.0+0x30d/0x1220 __netif_receive_skb_list_core+0xfc/0x250 netif_receive_skb_list_internal+0x20c/0x3d0 ? eth_type_trans+0x137/0x160 netif_receive_skb_list+0x25/0x140Quoted 9 lines
xdp_test_run_batch.constprop.0+0x65b/0x6e0 bpf_test_run_xdp_live+0x1ec/0x3b0 bpf_prog_test_run_xdp+0x49d/0x6e0 __sys_bpf+0x446/0x27b0 __x64_sys_bpf+0x1a/0x30 x64_sys_call+0x146c/0x26e0 do_syscall_64+0xd3/0x1510 entry_SYSCALL_64_after_hwframe+0x76/0x7e
Quoted 4 lines
Please, can you always provide symbols in such traces ? You can use
scripts/decode_stacktrace.shto make the trace really nice, instead of ugly.
Hi Eric,
Thank you for the suggestion. I didn't include the fully decoded stack trace in the cover letter because the syzkaller report already contains the complete information. You can find it here if needed:
https://syzkaller.appspot.com/text?tag=CrashReport&x=15448952580000
https://syzkaller.appspot.com/bug?extid=80e046b8da2820b6ba73
Quoted 53 lines · Jiayuan Chen
Quoted 43 lines · Eric Dumazet
Quoted 31 lines · Jiayuan Chen
syzkaller reported a kernel panic [1] with the following crash stack:
Call Trace: BUG: unable to handle page fault for address: ffff8ebd08580000 PF: supervisor write access in kernel mode PF: error_code(0x0002) - not-present page PGD 11f201067 P4D 11f201067 PUD 0 Oops: Oops: 0002 [#1] SMP PTI CPU: 2 UID: 0 PID: 451 Comm: test_progs Not tainted 6.19.0+ #161 PREEMPT_RT RIP: 0010:bond_rr_gen_slave_id+0x90/0xd0 RSP: 0018:ffffd3f4815f3448 EFLAGS: 00010246 RAX: 0000000000000001 RBX: 0000000000000001 RCX: ffff8ebc8728b17e RDX: 0000000000000000 RSI: ffffd3f4815f3538 RDI: ffff8ebc8abcce40 RBP: ffffd3f4815f3460 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000000 R12: ffffd3f4815f3538 R13: ffff8ebc8abcce40 R14: ffff8ebc8728b17f R15: ffff8ebc8728b170 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: ffff8ebd08580000 CR3: 000000010a808006 CR4: 0000000000770ef0 PKRU: 55555554 Call Trace: <TASK> bond_xdp_get_xmit_slave+0xc0/0x240 xdp_master_redirect+0x74/0xc0 bpf_prog_run_generic_xdp+0x2f2/0x3f0 do_xdp_generic+0x1fd/0x3d0 __netif_receive_skb_core.constprop.0+0x30d/0x1220 __netif_receive_skb_list_core+0xfc/0x250 netif_receive_skb_list_internal+0x20c/0x3d0 ? eth_type_trans+0x137/0x160 netif_receive_skb_list+0x25/0x140Quoted 9 lines
xdp_test_run_batch.constprop.0+0x65b/0x6e0 bpf_test_run_xdp_live+0x1ec/0x3b0 bpf_prog_test_run_xdp+0x49d/0x6e0 __sys_bpf+0x446/0x27b0 __x64_sys_bpf+0x1a/0x30 x64_sys_call+0x146c/0x26e0 do_syscall_64+0xd3/0x1510 entry_SYSCALL_64_after_hwframe+0x76/0x7eQuoted 4 lines
Please, can you always provide symbols in such traces ? You can use
scripts/decode_stacktrace.shto make the trace really nice, instead of ugly.
Quoted 8 lines
Hi Eric,
Thank you for the suggestion. I didn't include the fully decoded stack trace in the cover letter because the syzkaller report already contains the complete information. You can find it here if needed:
https://syzkaller.appspot.com/text?tag=CrashReport&x=15448952580000
https://syzkaller.appspot.com/bug?extid=80e046b8da2820b6ba73
Exactly.
Either copy the syzbot stack traces when they have the symbols,
or do not copy them if they don't have them, a link to them is just good enough.
Re: [PATCH net v5 1/2] bonding: fix null-ptr-deref in bond_rr_gen_slave_id()
In reply to Jiayuan ChenQuoted 23 lines · Jiayuan Chen
From: Jiayuan Chen <[email protected]>
bond_rr_gen_slave_id()dereferencesbond->rr_tx_counterwithout a NULL check.rr_tx_counteris a per-CPU counter only allocated inbond_open()when the bond mode is round-robin. If the bond device was never brought up,rr_tx_counterremains NULL, causing a null-ptr-deref.The XDP redirect path can reach this code even when the bond is not up:
bpf_master_redirect_enabled_keyis a global static key, so when any bond device has native XDP attached, theXDP_TX->xdp_master_redirect()interception is enabled for all bond slaves system-wide. This allows the pathxdp_master_redirect()->bond_xdp_get_xmit_slave()->bond_xdp_xmit_roundrobin_slave_get()->bond_rr_gen_slave_id()to be reached on a bond that was never opened.Fix this by adding a NULL check with
unlikely()inbond_rr_gen_slave_id()before dereferencingrr_tx_counter. Whenrr_tx_counteris NULL (bond was never opened), fall back toget_random_u32()for slave selection. The allocation inbond_open()is kept, withWRITE_ONCE()added to safely publish the pointer to the XDP read side. A plain read suffices for the !bond->rr_tx_counter guard inbond_open()itself, asbond_open()runs under RTNL lock and is the only writer ofrr_tx_counter.
Quoted 8 lines
Fixes 879af96ffd72 ("net, core: Add support for XDP redirection to slave device")Reported-by [email protected]Signed-off-by Jiayuan Chen <[email protected]>--- drivers/net/bonding/bond_main.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
This is Jay's patch + the unlikely change, looks good to me.
Cheers,
Nik
Quoted 31 lines
drivers/net/bonding/bond_main.c +7
-2@@ -4290,9 +4290,11 @@ static int bond_open(struct net_device *bond_dev)struct slave *slave;if (BOND_MODE (bond) ==BOND_MODE_ROUNDROBIN && !bond->rr_tx_counter) { - bond->rr_tx_counter =alloc_percpu (u32); -if (!bond->rr_tx_counter) +u32 __percpu *rr_tx_tmp =alloc_percpu (u32); + +if (!rr_tx_tmp)return -ENOMEM ; +WRITE_ONCE (bond->rr_tx_counter, rr_tx_tmp); }/* reset slave->backup and slave->inactive */ @@ -4883,6 +4885,9 @@ static u32 bond_rr_gen_slave_id(struct bonding *bond)struct reciprocal_value reciprocal_packets_per_slave;int packets_per_slave = bond->params.packets_per_slave; +if (unlikely (!READ_ONCE (bond->rr_tx_counter))) +return get_random_u32 (); +switch (packets_per_slave) {case 0 : slave_id =get_random_u32 ();Signature · 1 line
2.43.0
Re: [PATCH net v5 1/2] bonding: fix null-ptr-deref in bond_rr_gen_slave_id()
Quoted 34 lines · Nikolay Aleksandrov
Quoted 23 lines · Jiayuan Chen
From: Jiayuan Chen <[email protected]>
bond_rr_gen_slave_id()dereferencesbond->rr_tx_counterwithout a NULL check.rr_tx_counteris a per-CPU counter only allocated inbond_open()when the bond mode is round-robin. If the bond device was never brought up,rr_tx_counterremains NULL, causing a null-ptr-deref.The XDP redirect path can reach this code even when the bond is not up:
bpf_master_redirect_enabled_keyis a global static key, so when any bond device has native XDP attached, theXDP_TX->xdp_master_redirect()interception is enabled for all bond slaves system-wide. This allows the pathxdp_master_redirect()->bond_xdp_get_xmit_slave()->bond_xdp_xmit_roundrobin_slave_get()->bond_rr_gen_slave_id()to be reached on a bond that was never opened.Fix this by adding a NULL check with
unlikely()inbond_rr_gen_slave_id()before dereferencingrr_tx_counter. Whenrr_tx_counteris NULL (bond was never opened), fall back toget_random_u32()for slave selection. The allocation inbond_open()is kept, withWRITE_ONCE()added to safely publish the pointer to the XDP read side. A plain read suffices for the !bond->rr_tx_counter guard inbond_open()itself, asbond_open()runs under RTNL lock and is the only writer ofrr_tx_counter.Quoted 8 lines
Fixes 879af96ffd72 ("net, core: Add support for XDP redirection to slave device")Reported-by [email protected]Signed-off-by Jiayuan Chen <[email protected]>--- drivers/net/bonding/bond_main.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
Quoted 2 lines
This is Jay's patch + the unlikely change, looks good to me.
Reviewed-by Nikolay Aleksandrov <[email protected]>
Orthogonal to this patch :
get_random_u32() typical cost is around 10 to 20 ns, I really wonder
if this makes sense
for the packets_per_slave == 0 or 1 case to haves this kind of randomness in the first place.
Perhaps we could use a
static DEFINE_PER_CPU(u32, rr_tx_counter)
And :
slave_id = this_cpu_inc_return(rr_tx_counter);
Re: [PATCH net v5 1/2] bonding: fix null-ptr-deref in bond_rr_gen_slave_id()
Quoted 47 lines · Eric Dumazet
Quoted 34 lines · Nikolay Aleksandrov
Quoted 23 lines · Jiayuan Chen
From: Jiayuan Chen <[email protected]>
bond_rr_gen_slave_id()dereferencesbond->rr_tx_counterwithout a NULL check.rr_tx_counteris a per-CPU counter only allocated inbond_open()when the bond mode is round-robin. If the bond device was never brought up,rr_tx_counterremains NULL, causing a null-ptr-deref.The XDP redirect path can reach this code even when the bond is not up:
bpf_master_redirect_enabled_keyis a global static key, so when any bond device has native XDP attached, theXDP_TX->xdp_master_redirect()interception is enabled for all bond slaves system-wide. This allows the pathxdp_master_redirect()->bond_xdp_get_xmit_slave()->bond_xdp_xmit_roundrobin_slave_get()->bond_rr_gen_slave_id()to be reached on a bond that was never opened.Fix this by adding a NULL check with
unlikely()inbond_rr_gen_slave_id()before dereferencingrr_tx_counter. Whenrr_tx_counteris NULL (bond was never opened), fall back toget_random_u32()for slave selection. The allocation inbond_open()is kept, withWRITE_ONCE()added to safely publish the pointer to the XDP read side. A plain read suffices for the !bond->rr_tx_counter guard inbond_open()itself, asbond_open()runs under RTNL lock and is the only writer ofrr_tx_counter.Quoted 8 lines
Fixes 879af96ffd72 ("net, core: Add support for XDP redirection to slave device")Reported-by [email protected]Signed-off-by Jiayuan Chen <[email protected]>--- drivers/net/bonding/bond_main.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)Quoted 2 lines
This is Jay's patch + the unlikely change, looks good to me.
Reviewed-by Nikolay Aleksandrov <[email protected]>Orthogonal to this patch :
get_random_u32()typical cost is around 10 to 20 ns, I really wonder
if this makes sense
for thepackets_per_slave== 0 or 1 case to haves this kind of randomness in the first place.
Quoted 6 lines
Perhaps we could use a
static
DEFINE_PER_CPU(u32, rr_tx_counter)And :
slave_id=this_cpu_inc_return(rr_tx_counter);
I also have mixed feelings about this patch.
We probably should detect that the device is not ready before hitting something deeper in the stack.
Sure, a NULL deref is avoided, bu what happens next ?
We send a packet while the device is not UP, I am pretty sure this violates at least some RCU rules in device dismantling.
Re: [PATCH net v5 1/2] bonding: fix null-ptr-deref in bond_rr_gen_slave_id()
Quoted 57 lines · Eric Dumazet
Quoted 47 lines · Eric Dumazet
Quoted 34 lines · Nikolay Aleksandrov
Quoted 23 lines · Jiayuan Chen
From: Jiayuan Chen <[email protected]>
bond_rr_gen_slave_id()dereferencesbond->rr_tx_counterwithout a NULL check.rr_tx_counteris a per-CPU counter only allocated inbond_open()when the bond mode is round-robin. If the bond device was never brought up,rr_tx_counterremains NULL, causing a null-ptr-deref.The XDP redirect path can reach this code even when the bond is not up:
bpf_master_redirect_enabled_keyis a global static key, so when any bond device has native XDP attached, theXDP_TX->xdp_master_redirect()interception is enabled for all bond slaves system-wide. This allows the pathxdp_master_redirect()->bond_xdp_get_xmit_slave()->bond_xdp_xmit_roundrobin_slave_get()->bond_rr_gen_slave_id()to be reached on a bond that was never opened.Fix this by adding a NULL check with
unlikely()inbond_rr_gen_slave_id()before dereferencingrr_tx_counter. Whenrr_tx_counteris NULL (bond was never opened), fall back toget_random_u32()for slave selection. The allocation inbond_open()is kept, withWRITE_ONCE()added to safely publish the pointer to the XDP read side. A plain read suffices for the !bond->rr_tx_counter guard inbond_open()itself, asbond_open()runs under RTNL lock and is the only writer ofrr_tx_counter.Quoted 8 lines
Fixes 879af96ffd72 ("net, core: Add support for XDP redirection to slave device")Reported-by [email protected]Signed-off-by Jiayuan Chen <[email protected]>--- drivers/net/bonding/bond_main.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)Quoted 2 lines
This is Jay's patch + the unlikely change, looks good to me.
Reviewed-by Nikolay Aleksandrov <[email protected]>Orthogonal to this patch :
get_random_u32()typical cost is around 10 to 20 ns, I really wonder
if this makes sense
for thepackets_per_slave== 0 or 1 case to haves this kind of randomness in the first place.Quoted 6 lines
Perhaps we could use a
static
DEFINE_PER_CPU(u32, rr_tx_counter)And :
slave_id=this_cpu_inc_return(rr_tx_counter);I also have mixed feelings about this patch.
Quoted 7 lines
We probably should detect that the device is not ready before hitting something deeper in the stack.
Sure, a NULL deref is avoided, bu what happens next ?
We send a packet while the device is not UP, I am pretty sure this violates at least some RCU rules in device dismantling.
IIRC when the redirect continues, the packet should get dropped if the device is not up (checks at a few places), but that's outside of bond's jurisdiction and after the slave id is needed in xdp master redirect's path unfortunately. I'm not sure it can reach much further, it just has the master dev's slave id generation in its path.
In any case we shouldn't crash in the slave id generation in the bonding, that ndo's only job is to return a slave id.
Paolo Abeni
maintainerRe: [PATCH net v5 1/2] bonding: fix null-ptr-deref in bond_rr_gen_slave_id()
Quoted 66 lines · Nikolay Aleksandrov
Quoted 57 lines · Eric Dumazet
Quoted 47 lines · Eric Dumazet
Quoted 34 lines · Nikolay Aleksandrov
Quoted 23 lines · Jiayuan Chen
From: Jiayuan Chen <[email protected]>
bond_rr_gen_slave_id()dereferencesbond->rr_tx_counterwithout a NULL check.rr_tx_counteris a per-CPU counter only allocated inbond_open()when the bond mode is round-robin. If the bond device was never brought up,rr_tx_counterremains NULL, causing a null-ptr-deref.The XDP redirect path can reach this code even when the bond is not up:
bpf_master_redirect_enabled_keyis a global static key, so when any bond device has native XDP attached, theXDP_TX->xdp_master_redirect()interception is enabled for all bond slaves system-wide. This allows the pathxdp_master_redirect()->bond_xdp_get_xmit_slave()->bond_xdp_xmit_roundrobin_slave_get()->bond_rr_gen_slave_id()to be reached on a bond that was never opened.Fix this by adding a NULL check with
unlikely()inbond_rr_gen_slave_id()before dereferencingrr_tx_counter. Whenrr_tx_counteris NULL (bond was never opened), fall back toget_random_u32()for slave selection. The allocation inbond_open()is kept, withWRITE_ONCE()added to safely publish the pointer to the XDP read side. A plain read suffices for the !bond->rr_tx_counter guard inbond_open()itself, asbond_open()runs under RTNL lock and is the only writer ofrr_tx_counter.Quoted 8 lines
Fixes 879af96ffd72 ("net, core: Add support for XDP redirection to slave device")Reported-by [email protected]Signed-off-by Jiayuan Chen <[email protected]>--- drivers/net/bonding/bond_main.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)Quoted 2 lines
This is Jay's patch + the unlikely change, looks good to me.
Reviewed-by Nikolay Aleksandrov <[email protected]>Orthogonal to this patch :
get_random_u32()typical cost is around 10 to 20 ns, I really wonder
if this makes sense
for thepackets_per_slave== 0 or 1 case to haves this kind of randomness in the first place.Quoted 6 lines
Perhaps we could use a
static
DEFINE_PER_CPU(u32, rr_tx_counter)And :
slave_id=this_cpu_inc_return(rr_tx_counter);I also have mixed feelings about this patch.
Quoted 7 lines
We probably should detect that the device is not ready before hitting something deeper in the stack.
Sure, a NULL deref is avoided, bu what happens next ?
We send a packet while the device is not UP, I am pretty sure this violates at least some RCU rules in device dismantling.
Quoted 8 lines
IIRC when the redirect continues, the packet should get dropped if the device is not up (checks at a few places), but that's outside of bond's jurisdiction and after the slave id is needed in xdp master redirect's path unfortunately. I'm not sure it can reach much further, it just has the master dev's slave id generation in its path.
In any case we shouldn't crash in the slave id generation in the bonding, that ndo's only job is to return a slave id.
I'm sorry for the back and forth, but I share Eric's concern. I think the approach suggested by Daniel:
https://lore.kernel.org/netdev/[email protected]/
or the initial patch form:
https://lore.kernel.org/netdev/[email protected]/T/#m7c67bb12f85bc88d583788fb6e41113c46208ae7
would be better. To respond to old concerns raised there: the check is IMHO bond-specific, as control moves from the lower interface to the upper bonding device, and the code is under an RCU critical section, the device can't go away before the xmit is completed.
/P
Re: [PATCH net v5 1/2] bonding: fix null-ptr-deref in bond_rr_gen_slave_id()
Quoted 76 lines · Paolo Abeni
Quoted 60 lines · Nikolay Aleksandrov
Quoted 52 lines · Eric Dumazet
Quoted 43 lines · Eric Dumazet
Quoted 32 lines · Nikolay Aleksandrov
Quoted 23 lines · Jiayuan Chen
From: Jiayuan Chen <[email protected]>
bond_rr_gen_slave_id()dereferencesbond->rr_tx_counterwithout a NULL check.rr_tx_counteris a per-CPU counter only allocated inbond_open()when the bond mode is round-robin. If the bond device was never brought up,rr_tx_counterremains NULL, causing a null-ptr-deref.The XDP redirect path can reach this code even when the bond is not up:
bpf_master_redirect_enabled_keyis a global static key, so when any bond device has native XDP attached, theXDP_TX->xdp_master_redirect()interception is enabled for all bond slaves system-wide. This allows the pathxdp_master_redirect()->bond_xdp_get_xmit_slave()->bond_xdp_xmit_roundrobin_slave_get()->bond_rr_gen_slave_id()to be reached on a bond that was never opened.Fix this by adding a NULL check with
unlikely()inbond_rr_gen_slave_id()before dereferencingrr_tx_counter. Whenrr_tx_counteris NULL (bond was never opened), fall back toget_random_u32()for slave selection. The allocation inbond_open()is kept, withWRITE_ONCE()added to safely publish the pointer to the XDP read side. A plain read suffices for the !bond->rr_tx_counter guard inbond_open()itself, asbond_open()runs under RTNL lock and is the only writer ofrr_tx_counter.Quoted 8 lines
Fixes 879af96ffd72 ("net, core: Add support for XDP redirection to slave device")Reported-by [email protected]Signed-off-by Jiayuan Chen <[email protected]>--- drivers/net/bonding/bond_main.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)Quoted 2 lines
This is Jay's patch + the unlikely change, looks good to me.
Reviewed-by Nikolay Aleksandrov <[email protected]>Orthogonal to this patch :
get_random_u32()typical cost is around 10 to 20 ns, I really wonder
if this makes sense
for thepackets_per_slave== 0 or 1 case to haves this kind of randomness in the first place.Quoted 6 lines
Perhaps we could use a
static
DEFINE_PER_CPU(u32, rr_tx_counter)And :
slave_id=this_cpu_inc_return(rr_tx_counter);I also have mixed feelings about this patch.
Quoted 7 lines
We probably should detect that the device is not ready before hitting something deeper in the stack.
Sure, a NULL deref is avoided, bu what happens next ?
We send a packet while the device is not UP, I am pretty sure this violates at least some RCU rules in device dismantling.
Quoted 8 lines
IIRC when the redirect continues, the packet should get dropped if the device is not up (checks at a few places), but that's outside of bond's jurisdiction and after the slave id is needed in xdp master redirect's path unfortunately. I'm not sure it can reach much further, it just has the master dev's slave id generation in its path.
In any case we shouldn't crash in the slave id generation in the bonding, that ndo's only job is to return a slave id.
I'm sorry for the back and forth, but I share Eric's concern. I think the approach suggested by Daniel:
https://lore.kernel.org/netdev/[email protected]/
or the initial patch form:
Quoted 8 lines
https://lore.kernel.org/netdev/[email protected]/T/#m7c67bb12f85bc88d583788fb6e41113c46208ae7
would be better. To respond to old concerns raised there: the check is IMHO bond-specific, as control moves from the lower interface to the upper bonding device, and the code is under an RCU critical section, the device can't go away before the xmit is completed.
/P
Looking at this issue holistically:
1. The XDP layer fix addresses the root cause of the current issue
2. Adding a defensive null check in bond_rr_gen_slave_id() protects against buggy callers - whether from XDP or future code paths. This aligns with the defense-in-depth principle that Nikolay and Sebastian highlighted.
Could we include both in v1? This way, the bond layer is robust regardless
of who calls it, preventing similar crashes from other potential code paths.
Thanks
Re: [PATCH net v5 1/2] bonding: fix null-ptr-deref in bond_rr_gen_slave_id()
In reply to Paolo AbeniQuoted 73 lines · Paolo Abeni
Quoted 72 lines · Nikolay Aleksandrov
Quoted 57 lines · Eric Dumazet
Quoted 47 lines · Eric Dumazet
Quoted 34 lines · Nikolay Aleksandrov
Quoted 23 lines · Jiayuan Chen
From: Jiayuan Chen <[email protected]>
bond_rr_gen_slave_id()dereferencesbond->rr_tx_counterwithout a NULL check.rr_tx_counteris a per-CPU counter only allocated inbond_open()when the bond mode is round-robin. If the bond device was never brought up,rr_tx_counterremains NULL, causing a null-ptr-deref.The XDP redirect path can reach this code even when the bond is not up:
bpf_master_redirect_enabled_keyis a global static key, so when any bond device has native XDP attached, theXDP_TX->xdp_master_redirect()interception is enabled for all bond slaves system-wide. This allows the pathxdp_master_redirect()->bond_xdp_get_xmit_slave()->bond_xdp_xmit_roundrobin_slave_get()->bond_rr_gen_slave_id()to be reached on a bond that was never opened.Fix this by adding a NULL check with
unlikely()inbond_rr_gen_slave_id()before dereferencingrr_tx_counter. Whenrr_tx_counteris NULL (bond was never opened), fall back toget_random_u32()for slave selection. The allocation inbond_open()is kept, withWRITE_ONCE()added to safely publish the pointer to the XDP read side. A plain read suffices for the !bond->rr_tx_counter guard inbond_open()itself, asbond_open()runs under RTNL lock and is the only writer ofrr_tx_counter.Quoted 8 lines
Fixes 879af96ffd72 ("net, core: Add support for XDP redirection to slave device")Reported-by [email protected]Signed-off-by Jiayuan Chen <[email protected]>--- drivers/net/bonding/bond_main.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)Quoted 2 lines
This is Jay's patch + the unlikely change, looks good to me.
Reviewed-by Nikolay Aleksandrov <[email protected]>Orthogonal to this patch :
get_random_u32()typical cost is around 10 to 20 ns, I really wonder
if this makes sense
for thepackets_per_slave== 0 or 1 case to haves this kind of randomness in the first place.Quoted 6 lines
Perhaps we could use a
static
DEFINE_PER_CPU(u32, rr_tx_counter)And :
slave_id=this_cpu_inc_return(rr_tx_counter);I also have mixed feelings about this patch.
Quoted 7 lines
We probably should detect that the device is not ready before hitting something deeper in the stack.
Sure, a NULL deref is avoided, bu what happens next ?
We send a packet while the device is not UP, I am pretty sure this violates at least some RCU rules in device dismantling.
IIRC when the redirect continues, the packet should get dropped if the device is not up (checks at a few places), but that's outside of bond's jurisdiction and after the slave id is needed in xdp master redirect's path unfortunately. I'm not sure it can reach much further, it just has the master dev's slave id generation in its path.
Quoted 8 lines
Quoted 2 lines
In any case we shouldn't crash in the slave id generation in the bonding, that ndo's only job is to return a slave id.
I'm sorry for the back and forth, but I share Eric's concern. I think the approach suggested by Daniel:
That will work, I like Daniel's patch as well. It will add a test for all redirects for master devices, but I guess that is ok. For bonding it will work because the bond has the problem only while it was never opened (before first up).
IMO this patch still has value, because currently the code implicitly relies on a specific sequence of events, who knows tomorrow someone may find another way and again call that ndo while the bond is down.
Quoted 8 lines
or the initial patch form:
https://lore.kernel.org/netdev/[email protected]/T/#m7c67bb12f85bc88d583788fb6e41113c46208ae7
would be better. To respond to old concerns raised there: the check is IMHO bond-specific, as control moves from the lower interface to the upper bonding device, and the code is under an RCU critical section, the device can't go away before the xmit is completed.
This one I don't like, it is not about going away, it is more about adding 2 new tests for everyone and potentially 1 more cache line.
Quoted 3 lines
/P
Cheers,
Nik
Re: [PATCH net v5 1/2] bonding: fix null-ptr-deref in bond_rr_gen_slave_id()
In reply to Jiayuan ChenQuoted 1 line · Jiayuan Chen
f who calls it, preventing similar crashes from other potential code paths.
Sorry for the noise.
Could we include both:
1. Add link status check in xdp
2. Adding a defensive null check in bond_rr_gen_slave_id()
This way, the bond layer is robust regardless of who calls it, preventing similar crashes from other potential code paths.