Skip to contentSkip to conversation

bpf: Support non-linear skbs for BPF_PROG_TEST_RUN

PATCH Superseded Paul Chaignon 3 participants 9 messages Sep 14, 2025 – Sep 18, 2025

Subsystems (4)

v2 of 8 Superseded Latest is v8 →

lore
Share & tools
lore
#0 lore raw

v2 cover bpf: Support non-linear skbs for BPF_PROG_TEST_RUN

This patchset adds support for non-linear skbs when running tc programs with BPF_PROG_TEST_RUN.

We've had multiple bugs in the past few years in Cilium caused by missing calls to bpf_skb_pull_data(). Daniel suggested this new BPF_PROG_TEST_RUN flag as a way to uncover these bugs in our BPF tests.

Changes in v2:
- Made the linear size configurable via ctx->data_end, as suggested by Amery.
- Reworked the selftests to allow testing the configurable linear size.
- Fix warnings reported by kernel test robot on first commit.
- Rebased.

Paul Chaignon (4):
bpf: Refactor cleanup of bpf_prog_test_run_skb
bpf: Craft non-linear skbs in BPF_PROG_TEST_RUN selftests/bpf: Support non-linear flag in test loader selftests/bpf: Test direct packet access on non-linear skbs

 include/uapi/linux/bpf.h                      |   4 +
 net/bpf/test_run.c                            | 120 ++++++++++++------
 tools/include/uapi/linux/bpf.h                |   4 +
 tools/testing/selftests/bpf/progs/bpf_misc.h  |   4 +
 .../bpf/progs/verifier_direct_packet_access.c |  54 ++++++++
 tools/testing/selftests/bpf/test_loader.c     |  20 ++-
 6 files changed, 168 insertions(+), 38 deletions(-)
Signature · 1 line
2.43.0
#1 lore raw

PATCH v2 1/4 bpf: Refactor cleanup of bpf_prog_test_run_skb

This bit of refactoring aims to simplify the next patch in this series, in which freeing 'data' is a bit less straightforward.

---
 net/bpf/test_run.c | 26 ++++++++++++++------------
 1 file changed, 14 insertions(+), 12 deletions(-)
net/bpf/test_run.c +14 -12
index 4a862d605386..a9c81fec3290 100644
@@ -990,10 +990,10 @@ int bpf_prog_test_run_skb(struct bpf_prog *prog, const union bpf_attr *kattr,
 	u32 size = kattr->test.data_size_in;
 	u32 repeat = kattr->test.repeat;
 	struct __sk_buff *ctx = NULL;
+	struct sk_buff *skb = NULL;
+	struct sock *sk = NULL;
 	u32 retval, duration;
 	int hh_len = ETH_HLEN;
-	struct sk_buff *skb;
-	struct sock *sk;
 	void *data;
 	int ret;
 
@@ -1009,8 +1009,9 @@ int bpf_prog_test_run_skb(struct bpf_prog *prog, const union bpf_attr *kattr,
 
 	ctx = bpf_ctx_init(kattr, sizeof(struct __sk_buff));
 	if (IS_ERR(ctx)) {
-		kfree(data);
-		return PTR_ERR(ctx);
+		ret = PTR_ERR(ctx);
+		ctx = NULL;
+		goto out;
 	}
 
 	switch (prog->type) {
@@ -1030,24 +1031,23 @@ int bpf_prog_test_run_skb(struct bpf_prog *prog, const union bpf_attr *kattr,
 
 	sk = sk_alloc(net, AF_UNSPEC, GFP_USER, &bpf_dummy_proto, 1);
 	if (!sk) {
-		kfree(data);
-		kfree(ctx);
-		return -ENOMEM;
+		ret = -ENOMEM;
+		goto out;
 	}
 	sock_init_data(NULL, sk);
 
 	skb = slab_build_skb(data);
 	if (!skb) {
-		kfree(data);
-		kfree(ctx);
-		sk_free(sk);
-		return -ENOMEM;
+		ret = -ENOMEM;
+		goto out;
 	}
 	skb->sk = sk;
 
 	skb_reserve(skb, NET_SKB_PAD + NET_IP_ALIGN);
 	__skb_put(skb, size);
 
+	data = NULL; /* data released via kfree_skb */
+
 	if (ctx && ctx->ifindex > 1) {
 		dev = dev_get_by_index(net, ctx->ifindex);
 		if (!dev) {
@@ -1139,7 +1139,9 @@ int bpf_prog_test_run_skb(struct bpf_prog *prog, const union bpf_attr *kattr,
 	if (dev && dev != net->loopback_dev)
 		dev_put(dev);
 	kfree_skb(skb);
-	sk_free(sk);
+	kfree(data);
+	if (sk)
+		sk_free(sk);
 	kfree(ctx);
 	return ret;
 }
Signature · 1 line
2.43.0
#2 lore raw

PATCH v2 2/4 bpf: Craft non-linear skbs in BPF_PROG_TEST_RUN

In reply to Paul Chaignon

This patch adds support for crafting non-linear skbs in BPF test runs for tc programs, via a new flag BPF_F_TEST_SKB_NON_LINEAR. When this flag is set, the size of the linear area is given by ctx->data_end, with a minimum of ETH_HLEN always pulled in the linear area.

This is particularly useful to test support for non-linear skbs in large codebases such as Cilium. We've had multiple bugs in the past few years where we were missing calls to bpf_skb_pull_data(). This support in BPF_PROG_TEST_RUN would allow us to automatically cover this case in our BPF tests.

In addition to the selftests introduced later in the series, this patch was tested by setting BPF_F_TEST_SKB_NON_LINEAR for all tc selftests programs and checking test failures were expected.

---
 include/uapi/linux/bpf.h       |   4 ++
 net/bpf/test_run.c             | 102 ++++++++++++++++++++++++---------
 tools/include/uapi/linux/bpf.h |   4 ++
 3 files changed, 82 insertions(+), 28 deletions(-)
include/uapi/linux/bpf.h +4 -0
index 233de8677382..34272cd07dd2 100644
@@ -1448,6 +1448,10 @@ enum {
 #define BPF_F_TEST_XDP_LIVE_FRAMES	(1U << 1)
 /* If set, apply CHECKSUM_COMPLETE to skb and validate the checksum */
 #define BPF_F_TEST_SKB_CHECKSUM_COMPLETE	(1U << 2)
+/* If set, skb will be non-linear with the size of the linear area given
+ * by ctx.data_end or at least ETH_HLEN.
+ */
+#define BPF_F_TEST_SKB_NON_LINEAR	(1U << 3)
 
 /* type for BPF_ENABLE_STATS */
 enum bpf_stats_type {
net/bpf/test_run.c +74 -28

190 lines not on this page · raw message

tools/include/uapi/linux/bpf.h +4 -0
index 233de8677382..34272cd07dd2 100644
@@ -1448,6 +1448,10 @@ enum {
 #define BPF_F_TEST_XDP_LIVE_FRAMES	(1U << 1)
 /* If set, apply CHECKSUM_COMPLETE to skb and validate the checksum */
 #define BPF_F_TEST_SKB_CHECKSUM_COMPLETE	(1U << 2)
+/* If set, skb will be non-linear with the size of the linear area given
+ * by ctx.data_end or at least ETH_HLEN.
+ */
+#define BPF_F_TEST_SKB_NON_LINEAR	(1U << 3)
 
 /* type for BPF_ENABLE_STATS */
 enum bpf_stats_type {
Signature · 1 line
2.43.0
#6 lore raw

PATCH v2 3/4 selftests/bpf: Support non-linear flag in test loader

In reply to Paul Chaignon

This patch adds support for a new tag __linear_size in the test loader, to specify the size of the linear area in case of non-linear skbs. If the tag is absent or null, a linear skb is crafted.

---
 tools/testing/selftests/bpf/progs/bpf_misc.h  |  4 ++++
 .../bpf/progs/verifier_direct_packet_access.c |  1 +
 tools/testing/selftests/bpf/test_loader.c     | 20 +++++++++++++++++--
 3 files changed, 23 insertions(+), 2 deletions(-)
tools/testing/selftests/bpf/progs/bpf_misc.h +4 -0
index 7905396c9cc4..5406f1e76048 100644
@@ -119,6 +119,9 @@
  *                   Several __arch_* annotations could be specified at once.
  *                   When test case is not run on current arch it is marked as skipped.
  * __caps_unpriv     Specify the capabilities that should be set when running the test.
+ *
+ * __linear_size     Specify the size of the linear area of non-linear skbs, or
+ *                   0 for linear skbs.
  */
 #define __msg(msg)		__attribute__((btf_decl_tag("comment:test_expect_msg=" XSTR(__COUNTER__) "=" msg)))
 #define __xlated(msg)		__attribute__((btf_decl_tag("comment:test_expect_xlated=" XSTR(__COUNTER__) "=" msg)))
@@ -150,6 +153,7 @@
 #define __stderr_unpriv(msg)	__attribute__((btf_decl_tag("comment:test_expect_stderr_unpriv=" XSTR(__COUNTER__) "=" msg)))
 #define __stdout(msg)		__attribute__((btf_decl_tag("comment:test_expect_stdout=" XSTR(__COUNTER__) "=" msg)))
 #define __stdout_unpriv(msg)	__attribute__((btf_decl_tag("comment:test_expect_stdout_unpriv=" XSTR(__COUNTER__) "=" msg)))
+#define __linear_size(sz)	__attribute__((btf_decl_tag("comment:test_linear_size="XSTR(sz))))
 
 /* Define common capabilities tested using __caps_unpriv */
 #define CAP_NET_ADMIN		12
tools/testing/selftests/bpf/progs/verifier_direct_packet_access.c +1 -0
index 28b602ac9cbe..a61897e01a50 100644
@@ -1,6 +1,7 @@
 // SPDX-License-Identifier: GPL-2.0
 /* Converted from tools/testing/selftests/bpf/verifier/direct_packet_access.c */
 
+#include <linux/if_ether.h>
 #include <linux/bpf.h>
 #include <bpf/bpf_helpers.h>
 #include "bpf_misc.h"
tools/testing/selftests/bpf/test_loader.c +18 -2
index e065b467d509..18117973cd16 100644
@@ -42,6 +42,7 @@
 #define TEST_TAG_EXPECT_STDERR_PFX_UNPRIV "comment:test_expect_stderr_unpriv="
 #define TEST_TAG_EXPECT_STDOUT_PFX "comment:test_expect_stdout="
 #define TEST_TAG_EXPECT_STDOUT_PFX_UNPRIV "comment:test_expect_stdout_unpriv="
+#define TEST_TAG_LINEAR_SIZE "comment:test_linear_size="
 
 /* Warning: duplicated in bpf_misc.h */
 #define POINTER_VALUE	0xbadcafe
@@ -100,6 +101,7 @@ struct test_spec {
 	int mode_mask;
 	int arch_mask;
 	int load_mask;
+	int linear_sz;
 	bool auxiliary;
 	bool valid;
 };
@@ -632,6 +634,11 @@ static int parse_test_spec(struct test_loader *tester,
 					      &spec->unpriv.stdout);
 			if (err)
 				goto cleanup;
+		} else if (str_has_pfx(s, TEST_TAG_LINEAR_SIZE)) {
+			val = s + sizeof(TEST_TAG_LINEAR_SIZE) - 1;
+			err = parse_int(val, &spec->linear_sz, "test linear size");
+			if (err)
+				goto cleanup;
 		}
 	}
 
@@ -906,10 +913,11 @@ static bool is_unpriv_capable_map(struct bpf_map *map)
 	}
 }
 
-static int do_prog_test_run(int fd_prog, int *retval, bool empty_opts)
+static int do_prog_test_run(int fd_prog, int *retval, bool empty_opts, int linear_sz)
 {
 	__u8 tmp_out[TEST_DATA_LEN << 2] = {};
 	__u8 tmp_in[TEST_DATA_LEN] = {};
+	struct __sk_buff ctx = {};
 	int err, saved_errno;
 	LIBBPF_OPTS(bpf_test_run_opts, topts,
 		.data_in = tmp_in,
@@ -919,6 +927,13 @@ static int do_prog_test_run(int fd_prog, int *retval, bool empty_opts)
 		.repeat = 1,
 	);
 
+	if (linear_sz) {
+		topts.flags = BPF_F_TEST_SKB_NON_LINEAR;
+		ctx.data_end = linear_sz;
+		topts.ctx_in = &ctx;
+		topts.ctx_size_in = sizeof(ctx);
+	}
+
 	if (empty_opts) {
 		memset(&topts, 0, sizeof(struct bpf_test_run_opts));
 		topts.sz = sizeof(struct bpf_test_run_opts);
@@ -1168,7 +1183,8 @@ void run_subtest(struct test_loader *tester,
 		}
 
 		err = do_prog_test_run(bpf_program__fd(tprog), &retval,
-				       bpf_program__type(tprog) == BPF_PROG_TYPE_SYSCALL ? true : false);
+				       bpf_program__type(tprog) == BPF_PROG_TYPE_SYSCALL ? true : false,
+				       spec->linear_sz);
 		if (!err && retval != subspec->retval && subspec->retval != POINTER_VALUE) {
 			PRINT_FAIL("Unexpected retval: %d != %d\n", retval, subspec->retval);
 			goto tobj_cleanup;
Signature · 1 line
2.43.0
#7 lore raw

PATCH v2 4/4 selftests/bpf: Test direct packet access on non-linear skbs

In reply to Paul Chaignon

This patch adds new selftests in the direct packet access suite, to cover the non-linear case with BPF_F_TEST_SKB_NON_LINEAR. The three first tests cover the behavior of the bounds check with a non-linear skb (first two with min. linear size, third with long enough linear size). The last test adds a call to bpf_skb_pull_data() to be able to access the packet.

---
 .../bpf/progs/verifier_direct_packet_access.c | 53 +++++++++++++++++++
 1 file changed, 53 insertions(+)
tools/testing/selftests/bpf/progs/verifier_direct_packet_access.c +53 -0
index a61897e01a50..595b9cb904ea 100644
@@ -801,4 +801,57 @@ l0_%=:	/* exit(0) */					\
 	: __clobber_all);
 }
 
+#define access_test_non_linear(name, desc, retval, linear_sz)				\
+	SEC("tc")									\
+	__description("direct packet access: " #name " (non-linear, " desc ")")		\
+	__success __retval(retval)							\
+	__linear_size(linear_sz)							\
+	__naked void access_##name(void)						\
+	{										\
+		asm volatile ("								\
+		r2 = *(u32*)(r1 + %[skb_data]);						\
+		r3 = *(u32*)(r1 + %[skb_data_end]);					\
+		r0 = r2;								\
+		r0 += 22;								\
+		if r0 > r3 goto l0_%=;							\
+		r0 = *(u8*)(r0 - 1);							\
+		exit;									\
+	l0_%=:	r0 = 1;									\
+		exit;									\
+	"	:									\
+		: __imm_const(skb_data, offsetof(struct __sk_buff, data)),		\
+		  __imm_const(skb_data_end, offsetof(struct __sk_buff, data_end))	\
+		: __clobber_all);							\
+	}
+
+access_test_non_linear(test31, "too short", 1, ETH_HLEN);
+access_test_non_linear(test32, "too short", 1, 1);
+access_test_non_linear(test33, "long enough", 0, 22);
+
+SEC("tc")
+__description("direct packet access: test34 (non-linear, linearized)")
+__success __retval(0)
+__linear_size(ETH_HLEN)
+__naked void access_test34_non_linear_linearized(void)
+{
+	asm volatile ("					\
+	r6 = r1;					\
+	r2 = 22;					\
+	call %[bpf_skb_pull_data];			\
+	r2 = *(u32*)(r6 + %[skb_data]);			\
+	r3 = *(u32*)(r6 + %[skb_data_end]);		\
+	r0 = r2;					\
+	r0 += 22;					\
+	if r0 > r3 goto l0_%=;				\
+	r0 = *(u8*)(r0 - 1);				\
+	exit;						\
+l0_%=:	r0 = 1;						\
+	exit;						\
+"	:
+	: __imm(bpf_skb_pull_data),
+	  __imm_const(skb_data, offsetof(struct __sk_buff, data)),
+	  __imm_const(skb_data_end, offsetof(struct __sk_buff, data_end))
+	: __clobber_all);
+}
+
 char _license[] SEC("license") = "GPL";
Signature · 1 line
2.43.0
syzbot Tested-by [email protected] on the v2 series
#8 lore raw
In reply to Paul Chaignon
syzbot ci has tested the following series

syzbot ci has tested the following series

[v2] bpf: Support non-linear skbs for BPF_PROG_TEST_RUN
https://lore.kernel.org/all/[email protected]
* [PATCH bpf-next v2 1/4] bpf: Refactor cleanup of bpf_prog_test_run_skb
* [PATCH bpf-next v2 2/4] bpf: Craft non-linear skbs in BPF_PROG_TEST_RUN
* [PATCH bpf-next v2 3/4] selftests/bpf: Support non-linear flag in test loader
* [PATCH bpf-next v2 4/4] selftests/bpf: Test direct packet access on non-linear skbs

and found the following issue:
kernel BUG in bpf_test_init

Full report is available here:
https://ci.syzbot.org/series/23791187-8fc0-4c8a-bebc-3909e311c11e

***

kernel BUG in bpf_test_init

tree:      bpf-next
URL:       https://kernel.googlesource.com/pub/scm/linux/kernel/git/bpf/bpf-next.git
base:      a578b54a8ad282dd739e4d1f4e8352fc8ac1c4a0
arch:      amd64
compiler:  Debian clang version 20.1.8 (++20250708063551+0c9f909b7976-1~exp1~20250708183702.136), Debian LLD 20.1.8
config:    https://ci.syzbot.org/builds/037e1e74-be70-4216-ad05-430c2deea0bb/config
C repro:   https://ci.syzbot.org/findings/adf84c1c-28ba-48f5-9638-8339635cebe5/c_repro
syz repro: https://ci.syzbot.org/findings/adf84c1c-28ba-48f5-9638-8339635cebe5/syz_repro
------------[ cut here ]------------
kernel BUG at arch/x86/mm/physaddr.c:28!
Oops: invalid opcode: 0000 [#1] SMP KASAN PTI
CPU: 1 UID: 0 PID: 6014 Comm: syz.0.17 Not tainted syzkaller #0 PREEMPT(full) 
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.2-debian-1.16.2-1 04/01/2014
RIP: 0010:__phys_addr+0x173/0x180 arch/x86/mm/physaddr.c:28
Code: e8 02 2a 4b 00 48 c7 c7 50 d1 fa 8d 48 89 de 4c 89 f2 e8 e0 2f 8b 03 e9 4d ff ff ff e8 e6 29 4b 00 90 0f 0b e8 de 29 4b 00 90 <0f> 0b e8 d6 29 4b 00 90 0f 0b 0f 1f 00 90 90 90 90 90 90 90 90 90
RSP: 0018:ffffc900032d7b68 EFLAGS: 00010293
RAX: ffffffff81749722 RBX: 00006180047f3080 RCX: ffff8881079b9cc0
RDX: 0000000000000000 RSI: 0000000000000061 RDI: 0000000000000000
RBP: 00000000000000db R08: ffff88811fcc20da R09: 0000000000000000
R10: ffff88811fcc2000 R11: ffffed1023f9841c R12: 00000000000000db
R13: 00000000000000db R14: 0000000000000028 R15: 0000000000000061
FS:  000055557464c500(0000) GS:ffff8881a3c13000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 000000001f984000 CR4: 00000000000006f0
Call Trace:
 <TASK>
 virt_to_folio include/linux/mm.h:1180 [inline]
 kfree+0x77/0x440 mm/slub.c:4886
 bpf_test_init+0x1ba/0x220 net/bpf/test_run.c:686
 bpf_prog_test_run_skb+0x2c8/0x1700 net/bpf/test_run.c:1039
 bpf_prog_test_run+0x2c7/0x340 kernel/bpf/syscall.c:4590
 __sys_bpf+0x581/0x870 kernel/bpf/syscall.c:6047
 __do_sys_bpf kernel/bpf/syscall.c:6139 [inline]
 __se_sys_bpf kernel/bpf/syscall.c:6137 [inline]
 __x64_sys_bpf+0x7c/0x90 kernel/bpf/syscall.c:6137
 do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline]
 do_syscall_64+0xfa/0x3b0 arch/x86/entry/syscall_64.c:94
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f502198eba9
Code: ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 a8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007ffd90d34788 EFLAGS: 00000246 ORIG_RAX: 0000000000000141
RAX: ffffffffffffffda RBX: 00007f5021bd5fa0 RCX: 00007f502198eba9
RDX: 0000000000000050 RSI: 0000200000000240 RDI: 000000000000000a
RBP: 00007f5021a11e19 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 00007f5021bd5fa0 R14: 00007f5021bd5fa0 R15: 0000000000000003
 </TASK>
Modules linked in:
---[ end trace 0000000000000000 ]---
RIP: 0010:__phys_addr+0x173/0x180 arch/x86/mm/physaddr.c:28
Code: e8 02 2a 4b 00 48 c7 c7 50 d1 fa 8d 48 89 de 4c 89 f2 e8 e0 2f 8b 03 e9 4d ff ff ff e8 e6 29 4b 00 90 0f 0b e8 de 29 4b 00 90 <0f> 0b e8 d6 29 4b 00 90 0f 0b 0f 1f 00 90 90 90 90 90 90 90 90 90
RSP: 0018:ffffc900032d7b68 EFLAGS: 00010293
RAX: ffffffff81749722 RBX: 00006180047f3080 RCX: ffff8881079b9cc0
RDX: 0000000000000000 RSI: 0000000000000061 RDI: 0000000000000000
RBP: 00000000000000db R08: ffff88811fcc20da R09: 0000000000000000
R10: ffff88811fcc2000 R11: ffffed1023f9841c R12: 00000000000000db
R13: 00000000000000db R14: 0000000000000028 R15: 0000000000000061
FS:  000055557464c500(0000) GS:ffff8881a3c13000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 000000001f984000 CR4: 00000000000006f0

***

If these findings have caused you to resend the series or submit a separate fix, please add the following tag to your commit message:

---
This report is generated by a bot. It may contain errors. syzbot ci engineers can be reached at [email protected].

#3 lore raw

Re: [PATCH bpf-next v2 2/4] bpf: Craft non-linear skbs in BPF_PROG_TEST_RUN

In reply to Paul Chaignon
Quoted 52 lines · Paul Chaignon

This patch adds support for crafting non-linear skbs in BPF test runs for tc programs, via a new flag BPF_F_TEST_SKB_NON_LINEAR. When this flag is set, the size of the linear area is given by ctx->data_end, with a minimum of ETH_HLEN always pulled in the linear area.

This is particularly useful to test support for non-linear skbs in large codebases such as Cilium. We've had multiple bugs in the past few years where we were missing calls to bpf_skb_pull_data(). This support in BPF_PROG_TEST_RUN would allow us to automatically cover this case in our BPF tests.

In addition to the selftests introduced later in the series, this patch was tested by setting BPF_F_TEST_SKB_NON_LINEAR for all tc selftests programs and checking test failures were expected.

---
 include/uapi/linux/bpf.h       |   4 ++
 net/bpf/test_run.c             | 102 ++++++++++++++++++++++++---------
 tools/include/uapi/linux/bpf.h |   4 ++
 3 files changed, 82 insertions(+), 28 deletions(-)
include/uapi/linux/bpf.h +4 -0
index 233de8677382..34272cd07dd2 100644
@@ -1448,6 +1448,10 @@ enum {
 #define BPF_F_TEST_XDP_LIVE_FRAMES     (1U << 1)
 /* If set, apply CHECKSUM_COMPLETE to skb and validate the checksum */
 #define BPF_F_TEST_SKB_CHECKSUM_COMPLETE       (1U << 2)
+/* If set, skb will be non-linear with the size of the linear area given
+ * by ctx.data_end or at least ETH_HLEN.
+ */
+#define BPF_F_TEST_SKB_NON_LINEAR      (1U << 3)
 
 /* type for BPF_ENABLE_STATS */
 enum bpf_stats_type {
net/bpf/test_run.c +1 -2
index a9c81fec3290..aaa8f93d2fdb 100644
@@ -660,20 +660,29 @@ BTF_ID_FLAGS(func, bpf_kfunc_call_memb_release, KF_RELEASE)
 BTF_KFUNCS_END(test_sk_check_kfunc_ids)
 
 static void *bpf_test_init(const union bpf_attr *kattr, u32 user_size,
-                          u32 size, u32 headroom, u32 tailroom)
+                          u32 size, u32 headroom, u32 tailroom, bool nonlinear)
 {
        void __user *data_in = u64_to_user_ptr(kattr->test.data_in);
-       void *data;

net/bpf/test_run.c BTF_ID_FLAGS(func, bpf_kfunc_call_memb_release, KF_RELEASE) · line 673

Quoted 10 lines
net/bpf/test_run.c +4 -2
+       void *data, *dst;
 
        if (user_size < ETH_HLEN || user_size > PAGE_SIZE - headroom - tailroom)
                return ERR_PTR(-EINVAL);
 
-       size = SKB_DATA_ALIGN(size);
-       data = kzalloc(size + headroom + tailroom, GFP_USER);
+       /* In non-linear case, data_in is copied to the paged data */
+       if (nonlinear) {
+               data = alloc_page(GFP_USER);

Do we need more pages here for non-linear data larger than a page?

Quoted 4 lines
net/bpf/test_run.c +4 -0
+       } else {
+               size = SKB_DATA_ALIGN(size);
+               data = kzalloc(size + headroom + tailroom, GFP_USER);
+       }

net/bpf/test_run.c BTF_ID_FLAGS(func, bpf_kfunc_call_memb_release, KF_RELEASE) · line 686

Quoted 10 lines
net/bpf/test_run.c +5 -1
        if (!data)
                return ERR_PTR(-ENOMEM);
 
-       if (copy_from_user(data + headroom, data_in, user_size)) {
+       if (nonlinear)
+               dst = page_address(data);
+       else
+               dst = data + headroom;
+       if (copy_from_user(dst, data_in, user_size)) {
                kfree(data);

syzbot reported a bug. It seems like data allocated through alloc_page() got freed by kfree() here.

Quoted 67 lines
net/bpf/test_run.c +20 -15
                return ERR_PTR(-EFAULT);
        }
@@ -910,6 +919,12 @@ static int convert___skb_to_skb(struct sk_buff *skb, struct __sk_buff *__skb)
        /* cb is allowed */
 
        if (!range_is_zero(__skb, offsetofend(struct __sk_buff, cb),
+                          offsetof(struct __sk_buff, data_end)))
+               return -EINVAL;
+
+       /* data_end is allowed, but not copied to skb */
+
+       if (!range_is_zero(__skb, offsetofend(struct __sk_buff, data_end),
                           offsetof(struct __sk_buff, tstamp)))
                return -EINVAL;
 
@@ -984,7 +999,7 @@ static struct proto bpf_dummy_proto = {
 int bpf_prog_test_run_skb(struct bpf_prog *prog, const union bpf_attr *kattr,
                          union bpf_attr __user *uattr)
 {
-       bool is_l2 = false, is_direct_pkt_access = false;
+       bool is_l2 = false, is_direct_pkt_access = false, is_nonlinear = false;
        struct net *net = current->nsproxy->net_ns;
        struct net_device *dev = net->loopback_dev;
        u32 size = kattr->test.data_size_in;
@@ -994,25 +1009,14 @@ int bpf_prog_test_run_skb(struct bpf_prog *prog, const union bpf_attr *kattr,
        struct sock *sk = NULL;
        u32 retval, duration;
        int hh_len = ETH_HLEN;
+       int linear_size, ret;
        void *data;
-       int ret;
 
-       if ((kattr->test.flags & ~BPF_F_TEST_SKB_CHECKSUM_COMPLETE) ||
+       if ((kattr->test.flags & ~(BPF_F_TEST_SKB_CHECKSUM_COMPLETE | BPF_F_TEST_SKB_NON_LINEAR)) ||
            kattr->test.cpu || kattr->test.batch_size)
                return -EINVAL;
 
-       data = bpf_test_init(kattr, kattr->test.data_size_in,
-                            size, NET_SKB_PAD + NET_IP_ALIGN,
-                            SKB_DATA_ALIGN(sizeof(struct skb_shared_info)));
-       if (IS_ERR(data))
-               return PTR_ERR(data);
-
-       ctx = bpf_ctx_init(kattr, sizeof(struct __sk_buff));
-       if (IS_ERR(ctx)) {
-               ret = PTR_ERR(ctx);
-               ctx = NULL;
-               goto out;
-       }
+       is_nonlinear = kattr->test.flags & BPF_F_TEST_SKB_NON_LINEAR;
 
        switch (prog->type) {
        case BPF_PROG_TYPE_SCHED_CLS:
@@ -1029,6 +1033,27 @@ int bpf_prog_test_run_skb(struct bpf_prog *prog, const union bpf_attr *kattr,
                break;
        }
 
+       if (is_nonlinear && !is_l2)
+               return -EINVAL;
+
+       data = bpf_test_init(kattr, kattr->test.data_size_in,
+                            size, NET_SKB_PAD + NET_IP_ALIGN,
+                            SKB_DATA_ALIGN(sizeof(struct skb_shared_info)),
+                            is_nonlinear);
+       if (IS_ERR(data))
+               return PTR_ERR(data);
+

net/bpf/test_run.c int bpf_prog_test_run_skb(struct bpf_prog *prog, const union bpf_attr *kattr, · line 1055

Quoted 10 lines
net/bpf/test_run.c +10 -0
+       ctx = bpf_ctx_init(kattr, sizeof(struct __sk_buff));
+       if (IS_ERR(ctx)) {
+               ret = PTR_ERR(ctx);
+               ctx = NULL;
+               goto out;
+       }
+
+       linear_size = hh_len;
+       if (is_nonlinear && ctx && ctx->data_end > linear_size)
+               linear_size = ctx->data_end;

I think BPF_F_TEST_SKB_NON_LINEAR may not be necessary.

To not break backward compatibility (assuming existing users most likely zero initialized ctx), when ctx->data_end == 0 || ctx->data_end == data_size_in, allocate a linear skb as it used to be. Then, if ctx->data_end < data_size_in, allocate a non-linear skb.

WDYT?

Quoted 20 lines
net/bpf/test_run.c +8 -1
+
        sk = sk_alloc(net, AF_UNSPEC, GFP_USER, &bpf_dummy_proto, 1);
        if (!sk) {
                ret = -ENOMEM;
@@ -1036,15 +1061,32 @@ int bpf_prog_test_run_skb(struct bpf_prog *prog, const union bpf_attr *kattr,
        }
        sock_init_data(NULL, sk);
 
-       skb = slab_build_skb(data);
+       if (is_nonlinear)
+               skb = alloc_skb(NET_SKB_PAD + NET_IP_ALIGN + size +
+                               SKB_DATA_ALIGN(sizeof(struct skb_shared_info)),
+                               GFP_USER);
+       else
+               skb = slab_build_skb(data);
        if (!skb) {
                ret = -ENOMEM;
                goto out;
        }
+

net/bpf/test_run.c int bpf_prog_test_run_skb(struct bpf_prog *prog, const union bpf_attr *kattr, · line 1083

Quoted 10 lines
net/bpf/test_run.c +6 -1
        skb->sk = sk;
 
        skb_reserve(skb, NET_SKB_PAD + NET_IP_ALIGN);
-       __skb_put(skb, size);
+
+       if (is_nonlinear) {
+               skb_fill_page_desc(skb, 0, data, 0, size);
+               skb->truesize += PAGE_SIZE;
+               skb->data_len = size;
+               skb->len = size;

Do we need to update skb_shared_info?

Quoted 79 lines
net/bpf/test_run.c +16 -6
+
+               /* eth_type_trans expects the Ethernet header in the linear area. */
+               __pskb_pull_tail(skb, linear_size);
+       } else {
+               __skb_put(skb, size);
+       }
 
        data = NULL; /* data released via kfree_skb */
 
@@ -1127,9 +1169,11 @@ int bpf_prog_test_run_skb(struct bpf_prog *prog, const union bpf_attr *kattr,
        convert_skb_to___skb(skb, ctx);
 
        size = skb->len;
-       /* bpf program can never convert linear skb to non-linear */
-       if (WARN_ON_ONCE(skb_is_nonlinear(skb)))
+       if (skb_is_nonlinear(skb)) {
+               /* bpf program can never convert linear skb to non-linear */
+               WARN_ON_ONCE(!is_nonlinear);
                size = skb_headlen(skb);
+       }
        ret = bpf_test_finish(kattr, uattr, skb->data, NULL, size, retval,
                              duration);
        if (!ret)
@@ -1139,7 +1183,8 @@ int bpf_prog_test_run_skb(struct bpf_prog *prog, const union bpf_attr *kattr,
        if (dev && dev != net->loopback_dev)
                dev_put(dev);
        kfree_skb(skb);
-       kfree(data);
+       if (data)
+               is_nonlinear ? __free_page(data) : kfree(data);
        if (sk)
                sk_free(sk);
        kfree(ctx);
@@ -1265,7 +1310,7 @@ int bpf_prog_test_run_xdp(struct bpf_prog *prog, const union bpf_attr *kattr,
                size = max_data_sz;
        }
 
-       data = bpf_test_init(kattr, size, max_data_sz, headroom, tailroom);
+       data = bpf_test_init(kattr, size, max_data_sz, headroom, tailroom, false);
        if (IS_ERR(data)) {
                ret = PTR_ERR(data);
                goto free_ctx;
@@ -1388,7 +1433,7 @@ int bpf_prog_test_run_flow_dissector(struct bpf_prog *prog,
        if (size < ETH_HLEN)
                return -EINVAL;
 
-       data = bpf_test_init(kattr, kattr->test.data_size_in, size, 0, 0);
+       data = bpf_test_init(kattr, kattr->test.data_size_in, size, 0, 0, false);
        if (IS_ERR(data))
                return PTR_ERR(data);
 
@@ -1661,7 +1706,8 @@ int bpf_prog_test_run_nf(struct bpf_prog *prog,
 
        data = bpf_test_init(kattr, kattr->test.data_size_in, size,
                             NET_SKB_PAD + NET_IP_ALIGN,
-                            SKB_DATA_ALIGN(sizeof(struct skb_shared_info)));
+                            SKB_DATA_ALIGN(sizeof(struct skb_shared_info)),
+                            false);
        if (IS_ERR(data))
                return PTR_ERR(data);
 
tools/include/uapi/linux/bpf.h +4 -1
index 233de8677382..34272cd07dd2 100644
@@ -1448,6 +1448,10 @@ enum {
 #define BPF_F_TEST_XDP_LIVE_FRAMES     (1U << 1)
 /* If set, apply CHECKSUM_COMPLETE to skb and validate the checksum */
 #define BPF_F_TEST_SKB_CHECKSUM_COMPLETE       (1U << 2)
+/* If set, skb will be non-linear with the size of the linear area given
+ * by ctx.data_end or at least ETH_HLEN.
+ */
+#define BPF_F_TEST_SKB_NON_LINEAR      (1U << 3)
 
 /* type for BPF_ENABLE_STATS */
 enum bpf_stats_type {
--

2.43.0

#4 lore raw

Re: [PATCH bpf-next v2 2/4] bpf: Craft non-linear skbs in BPF_PROG_TEST_RUN

Thanks for the review Amery!

Quoted 1 line · Amery Hung

On Sun, Sep 14, 2025 at 8:10 AM Paul Chaignon <[email protected]> wrote:

[...]

Quoted 11 lines
Quoted 11 lines

static void *bpf_test_init(const union bpf_attr *kattr, u32 user_size,

net/bpf/test_run.c +2 -2
-                          u32 size, u32 headroom, u32 tailroom)
+                          u32 size, u32 headroom, u32 tailroom, bool nonlinear)
 {
        void __user *data_in = u64_to_user_ptr(kattr->test.data_in);
-       void *data;
+       void *data, *dst;
 
        if (user_size < ETH_HLEN || user_size > PAGE_SIZE - headroom - tailroom)
                return ERR_PTR(-EINVAL);
Quoted 7 lines

net/bpf/test_run.c BTF_ID_FLAGS(func, bpf_kfunc_call_memb_release, KF_RELEASE) · line 673

Quoted 5 lines
net/bpf/test_run.c +3 -2
-       size = SKB_DATA_ALIGN(size);
-       data = kzalloc(size + headroom + tailroom, GFP_USER);
+       /* In non-linear case, data_in is copied to the paged data */
+       if (nonlinear) {
+               data = alloc_page(GFP_USER);

Do we need more pages here for non-linear data larger than a page?

We're limiting user_size above to be at most PAGE_SIZE-headroom-tailroom, so I don't think we support more than a page of data. Am I missing something?

Quoted 16 lines
Quoted 5 lines
net/bpf/test_run.c +4 -0
+       } else {
+               size = SKB_DATA_ALIGN(size);
+               data = kzalloc(size + headroom + tailroom, GFP_USER);
+       }

net/bpf/test_run.c BTF_ID_FLAGS(func, bpf_kfunc_call_memb_release, KF_RELEASE) · line 686

Quoted 10 lines
net/bpf/test_run.c +5 -1
        if (!data)
                return ERR_PTR(-ENOMEM);
 
-       if (copy_from_user(data + headroom, data_in, user_size)) {
+       if (nonlinear)
+               dst = page_address(data);
+       else
+               dst = data + headroom;
+       if (copy_from_user(dst, data_in, user_size)) {
                kfree(data);
Quoted 2 lines

syzbot reported a bug. It seems like data allocated through alloc_page() got freed by kfree() here.

Yep, I've fixed it and it will be in the v3.

[...]

Quoted 27 lines
Quoted 14 lines
net/bpf/test_run.c +10 -0
@@ -1029,6 +1033,27 @@ int bpf_prog_test_run_skb(struct bpf_prog *prog, const union bpf_attr *kattr,
                break;
        }
 
+       if (is_nonlinear && !is_l2)
+               return -EINVAL;
+
+       data = bpf_test_init(kattr, kattr->test.data_size_in,
+                            size, NET_SKB_PAD + NET_IP_ALIGN,
+                            SKB_DATA_ALIGN(sizeof(struct skb_shared_info)),
+                            is_nonlinear);
+       if (IS_ERR(data))
+               return PTR_ERR(data);
+

net/bpf/test_run.c int bpf_prog_test_run_skb(struct bpf_prog *prog, const union bpf_attr *kattr, · line 1055

Quoted 10 lines
net/bpf/test_run.c +10 -0
+       ctx = bpf_ctx_init(kattr, sizeof(struct __sk_buff));
+       if (IS_ERR(ctx)) {
+               ret = PTR_ERR(ctx);
+               ctx = NULL;
+               goto out;
+       }
+
+       linear_size = hh_len;
+       if (is_nonlinear && ctx && ctx->data_end > linear_size)
+               linear_size = ctx->data_end;

I think BPF_F_TEST_SKB_NON_LINEAR may not be necessary.

Quoted 6 lines

To not break backward compatibility (assuming existing users most likely zero initialized ctx), when ctx->data_end == 0 || ctx->data_end == data_size_in, allocate a linear skb as it used to be. Then, if ctx->data_end < data_size_in, allocate a non-linear skb.

WDYT?

That makes sense, if only to be consistent with your patchset. It should be doable by just calling bpf_ctx_init before bpf_test_init. I'll try that.

Quoted 23 lines
Quoted 23 lines
net/bpf/test_run.c +8 -1
+
        sk = sk_alloc(net, AF_UNSPEC, GFP_USER, &bpf_dummy_proto, 1);
        if (!sk) {
                ret = -ENOMEM;
@@ -1036,15 +1061,32 @@ int bpf_prog_test_run_skb(struct bpf_prog *prog, const union bpf_attr *kattr,
        }
        sock_init_data(NULL, sk);
 
-       skb = slab_build_skb(data);
+       if (is_nonlinear)
+               skb = alloc_skb(NET_SKB_PAD + NET_IP_ALIGN + size +
+                               SKB_DATA_ALIGN(sizeof(struct skb_shared_info)),
+                               GFP_USER);
+       else
+               skb = slab_build_skb(data);
        if (!skb) {
                ret = -ENOMEM;
                goto out;
        }
+
        skb->sk = sk;
 
Quoted 10 lines

net/bpf/test_run.c int bpf_prog_test_run_skb(struct bpf_prog *prog, const union bpf_attr *kattr, · line 1083

Quoted 8 lines
net/bpf/test_run.c +6 -1
        skb_reserve(skb, NET_SKB_PAD + NET_IP_ALIGN);
-       __skb_put(skb, size);
+
+       if (is_nonlinear) {
+               skb_fill_page_desc(skb, 0, data, 0, size);
+               skb->truesize += PAGE_SIZE;
+               skb->data_len = size;
+               skb->len = size;

Do we need to update skb_shared_info?

skb_fill_page_desc() already does, at least the skb_shinfo(skb)->frags[0]. Do you have something else in mind?

[...]

#5 lore raw

Re: [PATCH bpf-next v2 2/4] bpf: Craft non-linear skbs in BPF_PROG_TEST_RUN

Quoted 31 lines · Paul Chaignon

Thanks for the review Amery!

Quoted 1 line · Amery Hung

On Sun, Sep 14, 2025 at 8:10 AM Paul Chaignon <[email protected]> wrote:

[...]

Quoted 11 lines
Quoted 11 lines

static void *bpf_test_init(const union bpf_attr *kattr, u32 user_size,

net/bpf/test_run.c +2 -2
-                          u32 size, u32 headroom, u32 tailroom)
+                          u32 size, u32 headroom, u32 tailroom, bool nonlinear)
 {
        void __user *data_in = u64_to_user_ptr(kattr->test.data_in);
-       void *data;
+       void *data, *dst;
 
        if (user_size < ETH_HLEN || user_size > PAGE_SIZE - headroom - tailroom)
                return ERR_PTR(-EINVAL);
Quoted 7 lines

net/bpf/test_run.c BTF_ID_FLAGS(func, bpf_kfunc_call_memb_release, KF_RELEASE) · line 673

Quoted 5 lines
net/bpf/test_run.c +3 -2
-       size = SKB_DATA_ALIGN(size);
-       data = kzalloc(size + headroom + tailroom, GFP_USER);
+       /* In non-linear case, data_in is copied to the paged data */
+       if (nonlinear) {
+               data = alloc_page(GFP_USER);

Do we need more pages here for non-linear data larger than a page?

We're limiting user_size above to be at most PAGE_SIZE-headroom-tailroom, so I don't think we support more than a page of data. Am I missing something?

[...]

Quoted 95 lines
Quoted 16 lines
Quoted 5 lines
net/bpf/test_run.c +4 -0
+       } else {
+               size = SKB_DATA_ALIGN(size);
+               data = kzalloc(size + headroom + tailroom, GFP_USER);
+       }

net/bpf/test_run.c BTF_ID_FLAGS(func, bpf_kfunc_call_memb_release, KF_RELEASE) · line 686

Quoted 10 lines
net/bpf/test_run.c +5 -1
        if (!data)
                return ERR_PTR(-ENOMEM);
 
-       if (copy_from_user(data + headroom, data_in, user_size)) {
+       if (nonlinear)
+               dst = page_address(data);
+       else
+               dst = data + headroom;
+       if (copy_from_user(dst, data_in, user_size)) {
                kfree(data);
Quoted 2 lines

syzbot reported a bug. It seems like data allocated through alloc_page() got freed by kfree() here.

Yep, I've fixed it and it will be in the v3.

[...]

Quoted 27 lines
Quoted 14 lines
net/bpf/test_run.c +10 -0
@@ -1029,6 +1033,27 @@ int bpf_prog_test_run_skb(struct bpf_prog *prog, const union bpf_attr *kattr,
                break;
        }
 
+       if (is_nonlinear && !is_l2)
+               return -EINVAL;
+
+       data = bpf_test_init(kattr, kattr->test.data_size_in,
+                            size, NET_SKB_PAD + NET_IP_ALIGN,
+                            SKB_DATA_ALIGN(sizeof(struct skb_shared_info)),
+                            is_nonlinear);
+       if (IS_ERR(data))
+               return PTR_ERR(data);
+

net/bpf/test_run.c int bpf_prog_test_run_skb(struct bpf_prog *prog, const union bpf_attr *kattr, · line 1055

Quoted 10 lines
net/bpf/test_run.c +10 -0
+       ctx = bpf_ctx_init(kattr, sizeof(struct __sk_buff));
+       if (IS_ERR(ctx)) {
+               ret = PTR_ERR(ctx);
+               ctx = NULL;
+               goto out;
+       }
+
+       linear_size = hh_len;
+       if (is_nonlinear && ctx && ctx->data_end > linear_size)
+               linear_size = ctx->data_end;

I think BPF_F_TEST_SKB_NON_LINEAR may not be necessary.

Quoted 6 lines

To not break backward compatibility (assuming existing users most likely zero initialized ctx), when ctx->data_end == 0 || ctx->data_end == data_size_in, allocate a linear skb as it used to be. Then, if ctx->data_end < data_size_in, allocate a non-linear skb.

WDYT?

That makes sense, if only to be consistent with your patchset. It should be doable by just calling bpf_ctx_init before bpf_test_init. I'll try that.

Quoted 23 lines
Quoted 23 lines
net/bpf/test_run.c +8 -1
+
        sk = sk_alloc(net, AF_UNSPEC, GFP_USER, &bpf_dummy_proto, 1);
        if (!sk) {
                ret = -ENOMEM;
@@ -1036,15 +1061,32 @@ int bpf_prog_test_run_skb(struct bpf_prog *prog, const union bpf_attr *kattr,
        }
        sock_init_data(NULL, sk);
 
-       skb = slab_build_skb(data);
+       if (is_nonlinear)
+               skb = alloc_skb(NET_SKB_PAD + NET_IP_ALIGN + size +
+                               SKB_DATA_ALIGN(sizeof(struct skb_shared_info)),
+                               GFP_USER);
+       else
+               skb = slab_build_skb(data);
        if (!skb) {
                ret = -ENOMEM;
                goto out;
        }
+
        skb->sk = sk;
 
Quoted 10 lines

net/bpf/test_run.c int bpf_prog_test_run_skb(struct bpf_prog *prog, const union bpf_attr *kattr, · line 1083

Quoted 8 lines
net/bpf/test_run.c +6 -1
        skb_reserve(skb, NET_SKB_PAD + NET_IP_ALIGN);
-       __skb_put(skb, size);
+
+       if (is_nonlinear) {
+               skb_fill_page_desc(skb, 0, data, 0, size);
+               skb->truesize += PAGE_SIZE;
+               skb->data_len = size;
+               skb->len = size;

Do we need to update skb_shared_info?

Quoted 3 lines

skb_fill_page_desc() already does, at least the skb_shinfo(skb)->frags[0]. Do you have something else in mind?

I mis-read bpf_test_init(). Thanks for the clarification!

Quoted 2 lines

[...]