bpf: Support non-linear skbs for BPF_PROG_TEST_RUN
Subsystems (4)
v2 cover bpf: Support non-linear skbs for BPF_PROG_TEST_RUN
This patchset adds support for non-linear skbs when running tc programs with BPF_PROG_TEST_RUN.
We've had multiple bugs in the past few years in Cilium caused by missing calls to bpf_skb_pull_data(). Daniel suggested this new BPF_PROG_TEST_RUN flag as a way to uncover these bugs in our BPF tests.
Changes in v2:
- Made the linear size configurable via ctx->data_end, as suggested by Amery.
- Reworked the selftests to allow testing the configurable linear size.
- Fix warnings reported by kernel test robot on first commit.
- Rebased.
Paul Chaignon (4):
bpf: Refactor cleanup of bpf_prog_test_run_skb
bpf: Craft non-linear skbs in BPF_PROG_TEST_RUN selftests/bpf: Support non-linear flag in test loader selftests/bpf: Test direct packet access on non-linear skbs
include/uapi/linux/bpf.h | 4 + net/bpf/test_run.c | 120 ++++++++++++------ tools/include/uapi/linux/bpf.h | 4 + tools/testing/selftests/bpf/progs/bpf_misc.h | 4 + .../bpf/progs/verifier_direct_packet_access.c | 54 ++++++++ tools/testing/selftests/bpf/test_loader.c | 20 ++- 6 files changed, 168 insertions(+), 38 deletions(-)
Signature · 1 line
2.43.0
PATCH v2 1/4 bpf: Refactor cleanup of bpf_prog_test_run_skb
This bit of refactoring aims to simplify the next patch in this series, in which freeing 'data' is a bit less straightforward.
--- net/bpf/test_run.c | 26 ++++++++++++++------------ 1 file changed, 14 insertions(+), 12 deletions(-)
net/bpf/test_run.c +14 -12
@@ -990,10 +990,10 @@ int bpf_prog_test_run_skb(struct bpf_prog *prog, const union bpf_attr *kattr,u32 size = kattr->test.data_size_in;u32 repeat = kattr->test.repeat;struct __sk_buff *ctx =NULL ; +struct sk_buff *skb =NULL ; +struct sock *sk =NULL ;u32 retval, duration;int hh_len =ETH_HLEN ; -struct sk_buff *skb; -struct sock *sk;void *data;int ret;
@@ -1009,8 +1009,9 @@ int bpf_prog_test_run_skb(struct bpf_prog *prog, const union bpf_attr *kattr, ctx =bpf_ctx_init (kattr,sizeof (struct __sk_buff ));if (IS_ERR (ctx)) { -kfree (data); -return PTR_ERR (ctx); + ret =PTR_ERR (ctx); + ctx =NULL ; + goto out; }switch (prog->type) {
@@ -1030,24 +1031,23 @@ int bpf_prog_test_run_skb(struct bpf_prog *prog, const union bpf_attr *kattr, sk =sk_alloc (net,AF_UNSPEC ,GFP_USER , &bpf_dummy_proto,1 );if (!sk) { -kfree (data); -kfree (ctx); -return -ENOMEM ; + ret = -ENOMEM ; + goto out; }sock_init_data (NULL , sk); skb =slab_build_skb (data);if (!skb) { -kfree (data); -kfree (ctx); -sk_free (sk); -return -ENOMEM ; + ret = -ENOMEM ; + goto out; } skb->sk = sk;skb_reserve (skb,NET_SKB_PAD +NET_IP_ALIGN );__skb_put (skb, size); + data =NULL ;/* data released via kfree_skb */ +if (ctx && ctx->ifindex >1 ) { dev =dev_get_by_index (net, ctx->ifindex);if (!dev) {
@@ -1139,7 +1139,9 @@ int bpf_prog_test_run_skb(struct bpf_prog *prog, const union bpf_attr *kattr,if (dev && dev != net->loopback_dev)dev_put (dev);kfree_skb (skb); -sk_free (sk); +kfree (data); +if (sk) +sk_free (sk);kfree (ctx);return ret; }
Signature · 1 line
2.43.0
PATCH v2 2/4 bpf: Craft non-linear skbs in BPF_PROG_TEST_RUN
In reply to Paul ChaignonThis patch adds support for crafting non-linear skbs in BPF test runs for tc programs, via a new flag BPF_F_TEST_SKB_NON_LINEAR. When this flag is set, the size of the linear area is given by ctx->data_end, with a minimum of ETH_HLEN always pulled in the linear area.
This is particularly useful to test support for non-linear skbs in large codebases such as Cilium. We've had multiple bugs in the past few years where we were missing calls to bpf_skb_pull_data(). This support in BPF_PROG_TEST_RUN would allow us to automatically cover this case in our BPF tests.
In addition to the selftests introduced later in the series, this patch was tested by setting BPF_F_TEST_SKB_NON_LINEAR for all tc selftests programs and checking test failures were expected.
--- include/uapi/linux/bpf.h | 4 ++ net/bpf/test_run.c | 102 ++++++++++++++++++++++++--------- tools/include/uapi/linux/bpf.h | 4 ++ 3 files changed, 82 insertions(+), 28 deletions(-)
include/uapi/linux/bpf.h +4 -0
@@ -1448,6 +1448,10 @@ enum {#define BPF_F_TEST_XDP_LIVE_FRAMES (1U << 1)/* If set, apply CHECKSUM_COMPLETE to skb and validate the checksum */ #define BPF_F_TEST_SKB_CHECKSUM_COMPLETE (1U << 2) +/* If set, skb will be non-linear with the size of the linear area given +* by ctx.data_end or at least ETH_HLEN. +*/ +#define BPF_F_TEST_SKB_NON_LINEAR (1U << 3)/* type for BPF_ENABLE_STATS */ enum bpf_stats_type {
net/bpf/test_run.c +74 -28
190 lines not on this page · raw message
tools/include/uapi/linux/bpf.h +4 -0
@@ -1448,6 +1448,10 @@ enum {#define BPF_F_TEST_XDP_LIVE_FRAMES (1U << 1)/* If set, apply CHECKSUM_COMPLETE to skb and validate the checksum */ #define BPF_F_TEST_SKB_CHECKSUM_COMPLETE (1U << 2) +/* If set, skb will be non-linear with the size of the linear area given +* by ctx.data_end or at least ETH_HLEN. +*/ +#define BPF_F_TEST_SKB_NON_LINEAR (1U << 3)/* type for BPF_ENABLE_STATS */ enum bpf_stats_type {
Signature · 1 line
2.43.0
PATCH v2 3/4 selftests/bpf: Support non-linear flag in test loader
In reply to Paul ChaignonThis patch adds support for a new tag __linear_size in the test loader, to specify the size of the linear area in case of non-linear skbs. If the tag is absent or null, a linear skb is crafted.
--- tools/testing/selftests/bpf/progs/bpf_misc.h | 4 ++++ .../bpf/progs/verifier_direct_packet_access.c | 1 + tools/testing/selftests/bpf/test_loader.c | 20 +++++++++++++++++-- 3 files changed, 23 insertions(+), 2 deletions(-)
tools/testing/selftests/bpf/progs/bpf_misc.h +4 -0
@@ -119,6 +119,9 @@* Several __arch_* annotations could be specified at once. * When test case is not run on current arch it is marked as skipped. * __caps_unpriv Specify the capabilities that should be set when running the test. +* +* __linear_size Specify the size of the linear area of non-linear skbs, or +* 0 for linear skbs. */ #define __msg (msg) __attribute__((btf_decl_tag("comment:test_expect_msg=" XSTR(__COUNTER__) "=" msg)))#define __xlated (msg) __attribute__((btf_decl_tag("comment:test_expect_xlated=" XSTR(__COUNTER__) "=" msg)))
@@ -150,6 +153,7 @@#define __stderr_unpriv (msg) __attribute__((btf_decl_tag("comment:test_expect_stderr_unpriv=" XSTR(__COUNTER__) "=" msg)))#define __stdout (msg) __attribute__((btf_decl_tag("comment:test_expect_stdout=" XSTR(__COUNTER__) "=" msg)))#define __stdout_unpriv (msg) __attribute__((btf_decl_tag("comment:test_expect_stdout_unpriv=" XSTR(__COUNTER__) "=" msg))) +#define __linear_size (sz) __attribute__((btf_decl_tag("comment:test_linear_size="XSTR(sz))))/* Define common capabilities tested using __caps_unpriv */ #define CAP_NET_ADMIN 12
tools/testing/selftests/bpf/progs/verifier_direct_packet_access.c +1 -0
@@ -1,6 +1,7 @@// SPDX-License-Identifier: GPL-2.0 /* Converted from tools/testing/selftests/bpf/verifier/direct_packet_access.c */ +#include <linux/if_ether.h> #include <linux/bpf.h> #include <bpf/bpf_helpers.h> #include "bpf_misc.h"
tools/testing/selftests/bpf/test_loader.c +18 -2
@@ -42,6 +42,7 @@#define TEST_TAG_EXPECT_STDERR_PFX_UNPRIV "comment:test_expect_stderr_unpriv="#define TEST_TAG_EXPECT_STDOUT_PFX "comment:test_expect_stdout="#define TEST_TAG_EXPECT_STDOUT_PFX_UNPRIV "comment:test_expect_stdout_unpriv=" +#define TEST_TAG_LINEAR_SIZE "comment:test_linear_size="/* Warning: duplicated in bpf_misc.h */ #define POINTER_VALUE 0xbadcafe
@@ -100,6 +101,7 @@ struct test_spec {int mode_mask;int arch_mask;int load_mask; +int linear_sz;bool auxiliary;bool valid; };
@@ -632,6 +634,11 @@ static int parse_test_spec(struct test_loader *tester, &spec->unpriv.stdout);if (err) goto cleanup; + }else if (str_has_pfx (s,TEST_TAG_LINEAR_SIZE )) { + val = s +sizeof (TEST_TAG_LINEAR_SIZE ) -1 ; + err =parse_int (val, &spec->linear_sz,"test linear size" ); +if (err) + goto cleanup; } }
@@ -906,10 +913,11 @@ static bool is_unpriv_capable_map(struct bpf_map *map) } } -static int do_prog_test_run (int fd_prog,int *retval,bool empty_opts) +static int do_prog_test_run (int fd_prog,int *retval,bool empty_opts,int linear_sz) {__u8 tmp_out[TEST_DATA_LEN <<2 ] = {};__u8 tmp_in[TEST_DATA_LEN ] = {}; +struct __sk_buff ctx = {};int err, saved_errno; LIBBPF_OPTS(bpf_test_run_opts, topts, .data_in = tmp_in,
@@ -919,6 +927,13 @@ static int do_prog_test_run(int fd_prog, int *retval, bool empty_opts) .repeat = 1, ); +if (linear_sz) { + topts.flags =BPF_F_TEST_SKB_NON_LINEAR ; + ctx.data_end = linear_sz; + topts.ctx_in = &ctx; + topts.ctx_size_in =sizeof (ctx); + } +if (empty_opts) {memset (&topts,0 ,sizeof (struct bpf_test_run_opts )); topts.sz =sizeof (struct bpf_test_run_opts );
@@ -1168,7 +1183,8 @@ void run_subtest(struct test_loader *tester, } err =do_prog_test_run (bpf_program__fd (tprog), &retval, -bpf_program__type (tprog) ==BPF_PROG_TYPE_SYSCALL ? true : false); +bpf_program__type (tprog) ==BPF_PROG_TYPE_SYSCALL ? true : false, + spec->linear_sz);if (!err && retval != subspec->retval && subspec->retval !=POINTER_VALUE ) {PRINT_FAIL ("Unexpected retval: %d != %d\n" , retval, subspec->retval); goto tobj_cleanup;
Signature · 1 line
2.43.0
PATCH v2 4/4 selftests/bpf: Test direct packet access on non-linear skbs
In reply to Paul ChaignonThis patch adds new selftests in the direct packet access suite, to cover the non-linear case with BPF_F_TEST_SKB_NON_LINEAR. The three first tests cover the behavior of the bounds check with a non-linear skb (first two with min. linear size, third with long enough linear size). The last test adds a call to bpf_skb_pull_data() to be able to access the packet.
--- .../bpf/progs/verifier_direct_packet_access.c | 53 +++++++++++++++++++ 1 file changed, 53 insertions(+)
tools/testing/selftests/bpf/progs/verifier_direct_packet_access.c +53 -0
@@ -801,4 +801,57 @@ l0_%=: /* exit(0) */ \ : __clobber_all); } +#define access_test_non_linear (name, desc, retval, linear_sz) \ + SEC("tc") \ + __description("direct packet access: " #name " (non-linear, " desc ")") \ + __success __retval(retval) \ + __linear_size(linear_sz) \ + __naked void access_##name(void) \ + { \ + asm volatile (" \ + r2 = *(u32*)(r1 + %[skb_data]); \ + r3 = *(u32*)(r1 + %[skb_data_end]); \ + r0 = r2; \ + r0 += 22; \ + if r0 > r3 goto l0_%=; \ + r0 = *(u8*)(r0 - 1); \ + exit; \ + l0_%=: r0 = 1; \ + exit; \ + " : \ + : __imm_const(skb_data, offsetof(struct __sk_buff, data)), \ + __imm_const(skb_data_end, offsetof(struct __sk_buff, data_end)) \ + : __clobber_all); \ + } + +access_test_non_linear (test31,"too short" ,1 ,ETH_HLEN ); +access_test_non_linear (test32,"too short" ,1 ,1 ); +access_test_non_linear (test33,"long enough" ,0 ,22 ); + +SEC("tc") +__description("direct packet access: test34 (non-linear, linearized)") +__success __retval(0) +__linear_size (ETH_HLEN) +__naked void access_test34_non_linear_linearized(void) +{ + asmvolatile (" \ +r6 = r1; \ +r2 = 22; \ +call %[bpf_skb_pull_data]; \ +r2 = *(u32*)(r6 + %[skb_data]); \ +r3 = *(u32*)(r6 + %[skb_data_end]); \ +r0 = r2; \ +r0 += 22; \ +if r0 > r3 goto l0_%=; \ +r0 = *(u8*)(r0 - 1); \ +exit; \ +l0_%=: r0 = 1; \ +exit; \ +" : + : __imm(bpf_skb_pull_data), +__imm_const (skb_data, offsetof(struct __sk_buff , data)), +__imm_const (skb_data_end, offsetof(struct __sk_buff , data_end)) + : __clobber_all); +} + char _license[] SEC("license") = "GPL";
Signature · 1 line
2.43.0
syzbot ci has tested the following series
syzbot ci has tested the following series
[v2] bpf: Support non-linear skbs for BPF_PROG_TEST_RUN
https://lore.kernel.org/all/[email protected]
* [PATCH bpf-next v2 1/4] bpf: Refactor cleanup of bpf_prog_test_run_skb
* [PATCH bpf-next v2 2/4] bpf: Craft non-linear skbs in BPF_PROG_TEST_RUN
* [PATCH bpf-next v2 3/4] selftests/bpf: Support non-linear flag in test loader
* [PATCH bpf-next v2 4/4] selftests/bpf: Test direct packet access on non-linear skbs
and found the following issue:
kernel BUG in bpf_test_init
Full report is available here:
https://ci.syzbot.org/series/23791187-8fc0-4c8a-bebc-3909e311c11e
***
kernel BUG in bpf_test_init
tree: bpf-next URL: https://kernel.googlesource.com/pub/scm/linux/kernel/git/bpf/bpf-next.git base: a578b54a8ad282dd739e4d1f4e8352fc8ac1c4a0 arch: amd64 compiler: Debian clang version 20.1.8 (++20250708063551+0c9f909b7976-1~exp1~20250708183702.136), Debian LLD 20.1.8 config: https://ci.syzbot.org/builds/037e1e74-be70-4216-ad05-430c2deea0bb/config C repro: https://ci.syzbot.org/findings/adf84c1c-28ba-48f5-9638-8339635cebe5/c_repro syz repro: https://ci.syzbot.org/findings/adf84c1c-28ba-48f5-9638-8339635cebe5/syz_repro
------------[ cut here ]------------ kernel BUG at arch/x86/mm/physaddr.c:28! Oops: invalid opcode: 0000 [#1] SMP KASAN PTI CPU: 1 UID: 0 PID: 6014 Comm: syz.0.17 Not tainted syzkaller #0 PREEMPT(full) Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.2-debian-1.16.2-1 04/01/2014 RIP: 0010:__phys_addr+0x173/0x180 arch/x86/mm/physaddr.c:28 Code: e8 02 2a 4b 00 48 c7 c7 50 d1 fa 8d 48 89 de 4c 89 f2 e8 e0 2f 8b 03 e9 4d ff ff ff e8 e6 29 4b 00 90 0f 0b e8 de 29 4b 00 90 <0f> 0b e8 d6 29 4b 00 90 0f 0b 0f 1f 00 90 90 90 90 90 90 90 90 90 RSP: 0018:ffffc900032d7b68 EFLAGS: 00010293 RAX: ffffffff81749722 RBX: 00006180047f3080 RCX: ffff8881079b9cc0 RDX: 0000000000000000 RSI: 0000000000000061 RDI: 0000000000000000 RBP: 00000000000000db R08: ffff88811fcc20da R09: 0000000000000000 R10: ffff88811fcc2000 R11: ffffed1023f9841c R12: 00000000000000db R13: 00000000000000db R14: 0000000000000028 R15: 0000000000000061 FS: 000055557464c500(0000) GS:ffff8881a3c13000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 000000001f984000 CR4: 00000000000006f0 Call Trace: <TASK> virt_to_folio include/linux/mm.h:1180 [inline] kfree+0x77/0x440 mm/slub.c:4886 bpf_test_init+0x1ba/0x220 net/bpf/test_run.c:686 bpf_prog_test_run_skb+0x2c8/0x1700 net/bpf/test_run.c:1039 bpf_prog_test_run+0x2c7/0x340 kernel/bpf/syscall.c:4590 __sys_bpf+0x581/0x870 kernel/bpf/syscall.c:6047 __do_sys_bpf kernel/bpf/syscall.c:6139 [inline] __se_sys_bpf kernel/bpf/syscall.c:6137 [inline] __x64_sys_bpf+0x7c/0x90 kernel/bpf/syscall.c:6137 do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline] do_syscall_64+0xfa/0x3b0 arch/x86/entry/syscall_64.c:94 entry_SYSCALL_64_after_hwframe+0x77/0x7f RIP: 0033:0x7f502198eba9 Code: ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 a8 ff ff ff f7 d8 64 89 01 48 RSP: 002b:00007ffd90d34788 EFLAGS: 00000246 ORIG_RAX: 0000000000000141 RAX: ffffffffffffffda RBX: 00007f5021bd5fa0 RCX: 00007f502198eba9 RDX: 0000000000000050 RSI: 0000200000000240 RDI: 000000000000000a RBP: 00007f5021a11e19 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000 R13: 00007f5021bd5fa0 R14: 00007f5021bd5fa0 R15: 0000000000000003 </TASK> Modules linked in: ---[ end trace 0000000000000000 ]--- RIP: 0010:__phys_addr+0x173/0x180 arch/x86/mm/physaddr.c:28 Code: e8 02 2a 4b 00 48 c7 c7 50 d1 fa 8d 48 89 de 4c 89 f2 e8 e0 2f 8b 03 e9 4d ff ff ff e8 e6 29 4b 00 90 0f 0b e8 de 29 4b 00 90 <0f> 0b e8 d6 29 4b 00 90 0f 0b 0f 1f 00 90 90 90 90 90 90 90 90 90 RSP: 0018:ffffc900032d7b68 EFLAGS: 00010293 RAX: ffffffff81749722 RBX: 00006180047f3080 RCX: ffff8881079b9cc0 RDX: 0000000000000000 RSI: 0000000000000061 RDI: 0000000000000000 RBP: 00000000000000db R08: ffff88811fcc20da R09: 0000000000000000 R10: ffff88811fcc2000 R11: ffffed1023f9841c R12: 00000000000000db R13: 00000000000000db R14: 0000000000000028 R15: 0000000000000061 FS: 000055557464c500(0000) GS:ffff8881a3c13000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000000 CR3: 000000001f984000 CR4: 00000000000006f0
***
If these findings have caused you to resend the series or submit a separate fix, please add the following tag to your commit message:
---
This report is generated by a bot. It may contain errors. syzbot ci engineers can be reached at [email protected].
Re: [PATCH bpf-next v2 2/4] bpf: Craft non-linear skbs in BPF_PROG_TEST_RUN
In reply to Paul ChaignonQuoted 52 lines · Paul Chaignon
This patch adds support for crafting non-linear skbs in BPF test runs for tc programs, via a new flag
BPF_F_TEST_SKB_NON_LINEAR. When this flag is set, the size of the linear area is given byctx->data_end, with a minimum ofETH_HLENalways pulled in the linear area.This is particularly useful to test support for non-linear skbs in large codebases such as Cilium. We've had multiple bugs in the past few years where we were missing calls to
bpf_skb_pull_data(). This support inBPF_PROG_TEST_RUNwould allow us to automatically cover this case in our BPF tests.In addition to the selftests introduced later in the series, this patch was tested by setting
BPF_F_TEST_SKB_NON_LINEARfor all tc selftests programs and checking test failures were expected.Suggested-by Daniel Borkmann <[email protected]>Signed-off-by Paul Chaignon <[email protected]>--- include/uapi/linux/bpf.h | 4 ++ net/bpf/test_run.c | 102 ++++++++++++++++++++++++--------- tools/include/uapi/linux/bpf.h | 4 ++ 3 files changed, 82 insertions(+), 28 deletions(-)include/uapi/linux/bpf.h +4
-0@@ -1448,6 +1448,10 @@ enum {#define BPF_F_TEST_XDP_LIVE_FRAMES (1U << 1)/* If set, apply CHECKSUM_COMPLETE to skb and validate the checksum */ #define BPF_F_TEST_SKB_CHECKSUM_COMPLETE (1U << 2) +/* If set, skb will be non-linear with the size of the linear area given +* by ctx.data_end or at least ETH_HLEN. +*/ +#define BPF_F_TEST_SKB_NON_LINEAR (1U << 3)/* type for BPF_ENABLE_STATS */ enum bpf_stats_type {net/bpf/test_run.c +1
-2@@ -660,20 +660,29 @@ BTF_ID_FLAGS(func, bpf_kfunc_call_memb_release, KF_RELEASE)BTF_KFUNCS_END (test_sk_check_kfunc_ids) static void *bpf_test_init(const union bpf_attr *kattr, u32 user_size, -u32 size,u32 headroom,u32 tailroom) +u32 size,u32 headroom,u32 tailroom,bool nonlinear) {void __user *data_in =u64_to_user_ptr (kattr->test.data_in); -void *data;
net/bpf/test_run.c BTF_ID_FLAGS(func, bpf_kfunc_call_memb_release, KF_RELEASE) · line 673
Quoted 10 lines
net/bpf/test_run.c +4
-2+void *data, *dst;if (user_size <ETH_HLEN || user_size >PAGE_SIZE - headroom - tailroom)return ERR_PTR (-EINVAL ); - size =SKB_DATA_ALIGN (size); - data =kzalloc (size + headroom + tailroom,GFP_USER ); +/* In non-linear case, data_in is copied to the paged data */ +if (nonlinear) { + data =alloc_page (GFP_USER );
Do we need more pages here for non-linear data larger than a page?
Quoted 4 lines
net/bpf/test_run.c +4
-0+ }else { + size =SKB_DATA_ALIGN (size); + data =kzalloc (size + headroom + tailroom,GFP_USER ); + }
net/bpf/test_run.c BTF_ID_FLAGS(func, bpf_kfunc_call_memb_release, KF_RELEASE) · line 686
Quoted 10 lines
net/bpf/test_run.c +5
-1if (!data)return ERR_PTR (-ENOMEM ); -if (copy_from_user (data + headroom, data_in, user_size)) { +if (nonlinear) + dst =page_address (data); +else + dst = data + headroom; +if (copy_from_user (dst, data_in, user_size)) {kfree (data);
syzbot reported a bug. It seems like data allocated through alloc_page() got freed by kfree() here.
Quoted 67 lines
net/bpf/test_run.c +20
-15return ERR_PTR (-EFAULT ); }@@ -910,6 +919,12 @@ static int convert___skb_to_skb(struct sk_buff *skb, struct __sk_buff *__skb)/* cb is allowed */ if (!range_is_zero(__skb, offsetofend(struct __sk_buff, cb), + offsetof(struct __sk_buff , data_end))) +return -EINVAL ; + +/* data_end is allowed, but not copied to skb */ + + if (!range_is_zero(__skb, offsetofend(struct __sk_buff, data_end), offsetof(struct __sk_buff , tstamp)))return -EINVAL ;@@ -984,7 +999,7 @@ static struct proto bpf_dummy_proto = {int bpf_prog_test_run_skb (struct bpf_prog *prog,const union bpf_attr *kattr,union bpf_attr __user *uattr) { -bool is_l2 = false, is_direct_pkt_access = false; +bool is_l2 = false, is_direct_pkt_access = false, is_nonlinear = false;struct net *net = current->nsproxy->net_ns;struct net_device *dev = net->loopback_dev;u32 size = kattr->test.data_size_in;@@ -994,25 +1009,14 @@ int bpf_prog_test_run_skb(struct bpf_prog *prog, const union bpf_attr *kattr,struct sock *sk =NULL ;u32 retval, duration;int hh_len =ETH_HLEN ; +int linear_size, ret;void *data; -int ret; -if ((kattr->test.flags & ~BPF_F_TEST_SKB_CHECKSUM_COMPLETE ) || +if ((kattr->test.flags & ~(BPF_F_TEST_SKB_CHECKSUM_COMPLETE |BPF_F_TEST_SKB_NON_LINEAR )) || kattr->test.cpu || kattr->test.batch_size)return -EINVAL ; - data =bpf_test_init (kattr, kattr->test.data_size_in, - size,NET_SKB_PAD +NET_IP_ALIGN , -SKB_DATA_ALIGN (sizeof (struct skb_shared_info ))); -if (IS_ERR (data)) -return PTR_ERR (data); - - ctx =bpf_ctx_init (kattr,sizeof (struct __sk_buff )); -if (IS_ERR (ctx)) { - ret =PTR_ERR (ctx); - ctx =NULL ; - goto out; - } + is_nonlinear = kattr->test.flags &BPF_F_TEST_SKB_NON_LINEAR ;switch (prog->type) {case BPF_PROG_TYPE_SCHED_CLS :@@ -1029,6 +1033,27 @@ int bpf_prog_test_run_skb(struct bpf_prog *prog, const union bpf_attr *kattr,break ; } +if (is_nonlinear && !is_l2) +return -EINVAL ; + + data =bpf_test_init (kattr, kattr->test.data_size_in, + size,NET_SKB_PAD +NET_IP_ALIGN , +SKB_DATA_ALIGN (sizeof (struct skb_shared_info )), + is_nonlinear); +if (IS_ERR (data)) +return PTR_ERR (data); +
net/bpf/test_run.c int bpf_prog_test_run_skb(struct bpf_prog *prog, const union bpf_attr *kattr, · line 1055
Quoted 10 lines
net/bpf/test_run.c +10
-0+ ctx =bpf_ctx_init (kattr,sizeof (struct __sk_buff )); +if (IS_ERR (ctx)) { + ret =PTR_ERR (ctx); + ctx =NULL ; + goto out; + } + + linear_size = hh_len; +if (is_nonlinear && ctx && ctx->data_end > linear_size) + linear_size = ctx->data_end;
I think BPF_F_TEST_SKB_NON_LINEAR may not be necessary.
To not break backward compatibility (assuming existing users most likely zero initialized ctx), when ctx->data_end == 0 || ctx->data_end == data_size_in, allocate a linear skb as it used to be. Then, if ctx->data_end < data_size_in, allocate a non-linear skb.
WDYT?
Quoted 20 lines
net/bpf/test_run.c +8
-1+ sk =sk_alloc (net,AF_UNSPEC ,GFP_USER , &bpf_dummy_proto,1 );if (!sk) { ret = -ENOMEM ;@@ -1036,15 +1061,32 @@ int bpf_prog_test_run_skb(struct bpf_prog *prog, const union bpf_attr *kattr, }sock_init_data (NULL , sk); - skb =slab_build_skb (data); +if (is_nonlinear) + skb =alloc_skb (NET_SKB_PAD +NET_IP_ALIGN + size + +SKB_DATA_ALIGN (sizeof (struct skb_shared_info )), +GFP_USER ); +else + skb =slab_build_skb (data);if (!skb) { ret = -ENOMEM ; goto out; } +
net/bpf/test_run.c int bpf_prog_test_run_skb(struct bpf_prog *prog, const union bpf_attr *kattr, · line 1083
Quoted 10 lines
net/bpf/test_run.c +6
-1skb->sk = sk;skb_reserve (skb,NET_SKB_PAD +NET_IP_ALIGN ); -__skb_put (skb, size); + +if (is_nonlinear) { +skb_fill_page_desc (skb,0 , data,0 , size); + skb->truesize +=PAGE_SIZE ; + skb->data_len = size; + skb->len = size;
Do we need to update skb_shared_info?
Quoted 79 lines
net/bpf/test_run.c +16
-6+ +/* eth_type_trans expects the Ethernet header in the linear area. */ +__pskb_pull_tail (skb, linear_size); + }else { +__skb_put (skb, size); + } data =NULL ;/* data released via kfree_skb */ @@ -1127,9 +1169,11 @@ int bpf_prog_test_run_skb(struct bpf_prog *prog, const union bpf_attr *kattr,convert_skb_to___skb (skb, ctx); size = skb->len; -/* bpf program can never convert linear skb to non-linear */ -if (WARN_ON_ONCE (skb_is_nonlinear (skb))) +if (skb_is_nonlinear (skb)) { +/* bpf program can never convert linear skb to non-linear */ +WARN_ON_ONCE (!is_nonlinear); size =skb_headlen (skb); + } ret =bpf_test_finish (kattr, uattr, skb->data,NULL , size, retval, duration);if (!ret)@@ -1139,7 +1183,8 @@ int bpf_prog_test_run_skb(struct bpf_prog *prog, const union bpf_attr *kattr,if (dev && dev != net->loopback_dev)dev_put (dev);kfree_skb (skb); -kfree (data); +if (data) + is_nonlinear ?__free_page (data) :kfree (data);if (sk)sk_free (sk);kfree (ctx);@@ -1265,7 +1310,7 @@ int bpf_prog_test_run_xdp(struct bpf_prog *prog, const union bpf_attr *kattr, size = max_data_sz; } - data =bpf_test_init (kattr, size, max_data_sz, headroom, tailroom); + data =bpf_test_init (kattr, size, max_data_sz, headroom, tailroom, false);if (IS_ERR (data)) { ret =PTR_ERR (data); goto free_ctx;@@ -1388,7 +1433,7 @@ int bpf_prog_test_run_flow_dissector(struct bpf_prog *prog,if (size <ETH_HLEN )return -EINVAL ; - data =bpf_test_init (kattr, kattr->test.data_size_in, size,0 ,0 ); + data =bpf_test_init (kattr, kattr->test.data_size_in, size,0 ,0 , false);if (IS_ERR (data))return PTR_ERR (data);@@ -1661,7 +1706,8 @@ int bpf_prog_test_run_nf(struct bpf_prog *prog, data =bpf_test_init (kattr, kattr->test.data_size_in, size,NET_SKB_PAD +NET_IP_ALIGN , -SKB_DATA_ALIGN (sizeof (struct skb_shared_info ))); +SKB_DATA_ALIGN (sizeof (struct skb_shared_info )), + false);if (IS_ERR (data))return PTR_ERR (data);tools/include/uapi/linux/bpf.h +4
-1@@ -1448,6 +1448,10 @@ enum {#define BPF_F_TEST_XDP_LIVE_FRAMES (1U << 1)/* If set, apply CHECKSUM_COMPLETE to skb and validate the checksum */ #define BPF_F_TEST_SKB_CHECKSUM_COMPLETE (1U << 2) +/* If set, skb will be non-linear with the size of the linear area given +* by ctx.data_end or at least ETH_HLEN. +*/ +#define BPF_F_TEST_SKB_NON_LINEAR (1U << 3)/* type for BPF_ENABLE_STATS */ enum bpf_stats_type { --2.43.0
Re: [PATCH bpf-next v2 2/4] bpf: Craft non-linear skbs in BPF_PROG_TEST_RUN
Thanks for the review Amery!
Quoted 1 line · Amery Hung
On Sun, Sep 14, 2025 at 8:10 AM Paul Chaignon <[email protected]> wrote:
[...]
Quoted 11 lines
Quoted 11 lines
static void *bpf_test_init(const union
bpf_attr*kattr, u32user_size,net/bpf/test_run.c +2
-2- u32 size, u32 headroom, u32 tailroom) + u32 size, u32 headroom, u32 tailroom, bool nonlinear) { void __user *data_in = u64_to_user_ptr(kattr->test.data_in); -void *data; +void *data, *dst;if (user_size <ETH_HLEN || user_size >PAGE_SIZE - headroom - tailroom)return ERR_PTR (-EINVAL );
Quoted 7 lines
net/bpf/test_run.c BTF_ID_FLAGS(func, bpf_kfunc_call_memb_release, KF_RELEASE) · line 673
Quoted 5 lines
net/bpf/test_run.c +3
-2- size =SKB_DATA_ALIGN (size); - data =kzalloc (size + headroom + tailroom,GFP_USER ); +/* In non-linear case, data_in is copied to the paged data */ +if (nonlinear) { + data =alloc_page (GFP_USER );Do we need more pages here for non-linear data larger than a page?
We're limiting user_size above to be at most PAGE_SIZE-headroom-tailroom, so I don't think we support more than a page of data. Am I missing something?
Quoted 16 lines
Quoted 5 lines
net/bpf/test_run.c +4
-0+ }else { + size =SKB_DATA_ALIGN (size); + data =kzalloc (size + headroom + tailroom,GFP_USER ); + }net/bpf/test_run.c BTF_ID_FLAGS(func, bpf_kfunc_call_memb_release, KF_RELEASE) · line 686
Quoted 10 lines
net/bpf/test_run.c +5
-1if (!data)return ERR_PTR (-ENOMEM ); -if (copy_from_user (data + headroom, data_in, user_size)) { +if (nonlinear) + dst =page_address (data); +else + dst = data + headroom; +if (copy_from_user (dst, data_in, user_size)) {kfree (data);
Quoted 2 lines
syzbot reported a bug. It seems like data allocated through
alloc_page()got freed bykfree()here.
Yep, I've fixed it and it will be in the v3.
[...]
Quoted 27 lines
Quoted 14 lines
net/bpf/test_run.c +10
-0@@ -1029,6 +1033,27 @@ int bpf_prog_test_run_skb(struct bpf_prog *prog, const union bpf_attr *kattr,break ; } +if (is_nonlinear && !is_l2) +return -EINVAL ; + + data =bpf_test_init (kattr, kattr->test.data_size_in, + size,NET_SKB_PAD +NET_IP_ALIGN , +SKB_DATA_ALIGN (sizeof (struct skb_shared_info )), + is_nonlinear); +if (IS_ERR (data)) +return PTR_ERR (data); +net/bpf/test_run.c int bpf_prog_test_run_skb(struct bpf_prog *prog, const union bpf_attr *kattr, · line 1055
Quoted 10 lines
net/bpf/test_run.c +10
-0+ ctx =bpf_ctx_init (kattr,sizeof (struct __sk_buff )); +if (IS_ERR (ctx)) { + ret =PTR_ERR (ctx); + ctx =NULL ; + goto out; + } + + linear_size = hh_len; +if (is_nonlinear && ctx && ctx->data_end > linear_size) + linear_size = ctx->data_end;I think
BPF_F_TEST_SKB_NON_LINEARmay not be necessary.
Quoted 6 lines
To not break backward compatibility (assuming existing users most likely zero initialized ctx), when
ctx->data_end== 0 ||ctx->data_end==data_size_in, allocate a linear skb as it used to be. Then, ifctx->data_end<data_size_in, allocate a non-linear skb.WDYT?
That makes sense, if only to be consistent with your patchset. It should be doable by just calling bpf_ctx_init before bpf_test_init. I'll try that.
Quoted 23 lines
Quoted 23 lines
net/bpf/test_run.c +8
-1+ sk =sk_alloc (net,AF_UNSPEC ,GFP_USER , &bpf_dummy_proto,1 );if (!sk) { ret = -ENOMEM ;@@ -1036,15 +1061,32 @@ int bpf_prog_test_run_skb(struct bpf_prog *prog, const union bpf_attr *kattr, }sock_init_data (NULL , sk); - skb =slab_build_skb (data); +if (is_nonlinear) + skb =alloc_skb (NET_SKB_PAD +NET_IP_ALIGN + size + +SKB_DATA_ALIGN (sizeof (struct skb_shared_info )), +GFP_USER ); +else + skb =slab_build_skb (data);if (!skb) { ret = -ENOMEM ; goto out; } + skb->sk = sk;
Quoted 10 lines
net/bpf/test_run.c int bpf_prog_test_run_skb(struct bpf_prog *prog, const union bpf_attr *kattr, · line 1083
Quoted 8 lines
net/bpf/test_run.c +6
-1skb_reserve (skb,NET_SKB_PAD +NET_IP_ALIGN ); -__skb_put (skb, size); + +if (is_nonlinear) { +skb_fill_page_desc (skb,0 , data,0 , size); + skb->truesize +=PAGE_SIZE ; + skb->data_len = size; + skb->len = size;Do we need to update
skb_shared_info?
skb_fill_page_desc() already does, at least the skb_shinfo(skb)->frags[0]. Do you have something else in mind?
[...]
Re: [PATCH bpf-next v2 2/4] bpf: Craft non-linear skbs in BPF_PROG_TEST_RUN
Quoted 31 lines · Paul Chaignon
Thanks for the review Amery!
Quoted 1 line · Amery Hung
On Sun, Sep 14, 2025 at 8:10 AM Paul Chaignon <[email protected]> wrote:
[...]
Quoted 11 lines
Quoted 11 lines
static void *bpf_test_init(const union
bpf_attr*kattr, u32user_size,net/bpf/test_run.c +2
-2- u32 size, u32 headroom, u32 tailroom) + u32 size, u32 headroom, u32 tailroom, bool nonlinear) { void __user *data_in = u64_to_user_ptr(kattr->test.data_in); -void *data; +void *data, *dst;if (user_size <ETH_HLEN || user_size >PAGE_SIZE - headroom - tailroom)return ERR_PTR (-EINVAL );Quoted 7 lines
net/bpf/test_run.c BTF_ID_FLAGS(func, bpf_kfunc_call_memb_release, KF_RELEASE) · line 673
Quoted 5 lines
net/bpf/test_run.c +3
-2- size =SKB_DATA_ALIGN (size); - data =kzalloc (size + headroom + tailroom,GFP_USER ); +/* In non-linear case, data_in is copied to the paged data */ +if (nonlinear) { + data =alloc_page (GFP_USER );Do we need more pages here for non-linear data larger than a page?
We're limiting
user_sizeabove to be at most PAGE_SIZE-headroom-tailroom, so I don't think we support more than a page of data. Am I missing something?
[...]
Quoted 95 lines
Quoted 16 lines
Quoted 5 lines
net/bpf/test_run.c +4
-0+ }else { + size =SKB_DATA_ALIGN (size); + data =kzalloc (size + headroom + tailroom,GFP_USER ); + }net/bpf/test_run.c BTF_ID_FLAGS(func, bpf_kfunc_call_memb_release, KF_RELEASE) · line 686
Quoted 10 lines
net/bpf/test_run.c +5
-1if (!data)return ERR_PTR (-ENOMEM ); -if (copy_from_user (data + headroom, data_in, user_size)) { +if (nonlinear) + dst =page_address (data); +else + dst = data + headroom; +if (copy_from_user (dst, data_in, user_size)) {kfree (data);Quoted 2 lines
syzbot reported a bug. It seems like data allocated through
alloc_page()got freed bykfree()here.Yep, I've fixed it and it will be in the v3.
[...]
Quoted 27 lines
Quoted 14 lines
net/bpf/test_run.c +10
-0@@ -1029,6 +1033,27 @@ int bpf_prog_test_run_skb(struct bpf_prog *prog, const union bpf_attr *kattr,break ; } +if (is_nonlinear && !is_l2) +return -EINVAL ; + + data =bpf_test_init (kattr, kattr->test.data_size_in, + size,NET_SKB_PAD +NET_IP_ALIGN , +SKB_DATA_ALIGN (sizeof (struct skb_shared_info )), + is_nonlinear); +if (IS_ERR (data)) +return PTR_ERR (data); +net/bpf/test_run.c int bpf_prog_test_run_skb(struct bpf_prog *prog, const union bpf_attr *kattr, · line 1055
Quoted 10 lines
net/bpf/test_run.c +10
-0+ ctx =bpf_ctx_init (kattr,sizeof (struct __sk_buff )); +if (IS_ERR (ctx)) { + ret =PTR_ERR (ctx); + ctx =NULL ; + goto out; + } + + linear_size = hh_len; +if (is_nonlinear && ctx && ctx->data_end > linear_size) + linear_size = ctx->data_end;I think
BPF_F_TEST_SKB_NON_LINEARmay not be necessary.Quoted 6 lines
To not break backward compatibility (assuming existing users most likely zero initialized ctx), when
ctx->data_end== 0 ||ctx->data_end==data_size_in, allocate a linear skb as it used to be. Then, ifctx->data_end<data_size_in, allocate a non-linear skb.WDYT?
That makes sense, if only to be consistent with your patchset. It should be doable by just calling
bpf_ctx_initbeforebpf_test_init. I'll try that.Quoted 23 lines
Quoted 23 lines
net/bpf/test_run.c +8
-1+ sk =sk_alloc (net,AF_UNSPEC ,GFP_USER , &bpf_dummy_proto,1 );if (!sk) { ret = -ENOMEM ;@@ -1036,15 +1061,32 @@ int bpf_prog_test_run_skb(struct bpf_prog *prog, const union bpf_attr *kattr, }sock_init_data (NULL , sk); - skb =slab_build_skb (data); +if (is_nonlinear) + skb =alloc_skb (NET_SKB_PAD +NET_IP_ALIGN + size + +SKB_DATA_ALIGN (sizeof (struct skb_shared_info )), +GFP_USER ); +else + skb =slab_build_skb (data);if (!skb) { ret = -ENOMEM ; goto out; } + skb->sk = sk;Quoted 10 lines
net/bpf/test_run.c int bpf_prog_test_run_skb(struct bpf_prog *prog, const union bpf_attr *kattr, · line 1083
Quoted 8 lines
net/bpf/test_run.c +6
-1skb_reserve (skb,NET_SKB_PAD +NET_IP_ALIGN ); -__skb_put (skb, size); + +if (is_nonlinear) { +skb_fill_page_desc (skb,0 , data,0 , size); + skb->truesize +=PAGE_SIZE ; + skb->data_len = size; + skb->len = size;Do we need to update
skb_shared_info?
Quoted 3 lines
skb_fill_page_desc()already does, at least the skb_shinfo(skb)->frags[0]. Do you have something else in mind?
I mis-read bpf_test_init(). Thanks for the clarification!
Quoted 2 lines
[...]