Skip to contentSkip to conversation

Introduce kfuncs for memory reads into dynptrs

PATCH Applied Mykyta Yatsenko 3 participants 8 messages May 12, 2025 – May 13, 2025

Subsystems (7)
Kselftest BPF Tracing BPF selftests +3 more subsystems
lore
Share & tools
lore
#0 lore raw

v5 cover Introduce kfuncs for memory reads into dynptrs

From: Mykyta Yatsenko <[email protected]>

This patch adds new kfuncs that enable reading variable-length user or kernel data directly into dynptrs.
These kfuncs provide a way to perform dynamically-sized reads while maintaining memory safety. Unlike existing bpf_probe_read_{user|kernel} APIs, which are limited to constant-sized reads, these new kfuncs allow for more flexible data access.

v4 -> v5
* Fix pointers annotations, use __user where necessary, cast where needed

v3 -> v4
* Added pid filtering in selftests

v2 -> v3
* Add KF_TRUSTED_ARGS for kfuncs that take pointer to task_struct as an argument
* Remove checks for non-NULL task, where it was not necessary
* Added comments on constants used in selftests, etc.

v1 -> v2
* Renaming helper functions to use "user_str" instead of "user_data_str" suffix

Mykyta Yatsenko (3):
helpers: make few bpf helpers public
bpf: implement dynptr copy kfuncs selftests/bpf: introduce tests for dynptr copy kfuncs

 include/linux/bpf.h                           |  14 ++
 kernel/bpf/helpers.c                          |  22 +-
 kernel/trace/bpf_trace.c                      | 193 +++++++++++++++
 tools/testing/selftests/bpf/DENYLIST          |   1 +
 .../testing/selftests/bpf/prog_tests/dynptr.c |  13 +
 .../selftests/bpf/progs/dynptr_success.c      | 230 ++++++++++++++++++
 6 files changed, 461 insertions(+), 12 deletions(-)
Signature · 1 line
2.49.0
#1 lore raw

PATCH v5 1/3 helpers: make few bpf helpers public

From: Mykyta Yatsenko <[email protected]>

Make bpf_dynptr_slice_rdwr, bpf_dynptr_check_off_len and __bpf_dynptr_write available outside of the helpers.c by adding their prototypes into linux/include/bpf.h. bpf_dynptr_check_off_len() implementation is moved to header and made inline explicitly, as small function should typically be inlined.

These functions are going to be used from bpf_trace.c in the next patch of this series.

---
 include/linux/bpf.h  | 14 ++++++++++++++
 kernel/bpf/helpers.c | 14 ++------------
 2 files changed, 16 insertions(+), 12 deletions(-)
include/linux/bpf.h +14 -0
index 3f0cc89c0622..83c56f40842b 100644
@@ -1349,6 +1349,20 @@ u32 __bpf_dynptr_size(const struct bpf_dynptr_kern *ptr);
 const void *__bpf_dynptr_data(const struct bpf_dynptr_kern *ptr, u32 len);
 void *__bpf_dynptr_data_rw(const struct bpf_dynptr_kern *ptr, u32 len);
 bool __bpf_dynptr_is_rdonly(const struct bpf_dynptr_kern *ptr);
+int __bpf_dynptr_write(const struct bpf_dynptr_kern *dst, u32 offset,
+		       void *src, u32 len, u64 flags);
+void *bpf_dynptr_slice_rdwr(const struct bpf_dynptr *p, u32 offset,
+			    void *buffer__opt, u32 buffer__szk);
+
+static inline int bpf_dynptr_check_off_len(const struct bpf_dynptr_kern *ptr, u32 offset, u32 len)
+{
+	u32 size = __bpf_dynptr_size(ptr);
+
+	if (len > size || offset > size - len)
+		return -E2BIG;
+
+	return 0;
+}
 
 #ifdef CONFIG_BPF_JIT
 int bpf_trampoline_link_prog(struct bpf_tramp_link *link,
kernel/bpf/helpers.c +2 -12
index e3a2662f4e33..048bd7ac1455 100644
@@ -1713,16 +1713,6 @@ void bpf_dynptr_set_null(struct bpf_dynptr_kern *ptr)
 	memset(ptr, 0, sizeof(*ptr));
 }
 
-static int bpf_dynptr_check_off_len(const struct bpf_dynptr_kern *ptr, u32 offset, u32 len)
-{
-	u32 size = __bpf_dynptr_size(ptr);
-
-	if (len > size || offset > size - len)
-		return -E2BIG;
-
-	return 0;
-}
-
 BPF_CALL_4(bpf_dynptr_from_mem, void *, data, u32, size, u64, flags, struct bpf_dynptr_kern *, ptr)
 {
 	int err;
@@ -1809,8 +1799,8 @@ static const struct bpf_func_proto bpf_dynptr_read_proto = {
 	.arg5_type	= ARG_ANYTHING,
 };
 
-static int __bpf_dynptr_write(const struct bpf_dynptr_kern *dst, u32 offset, void *src,
-			      u32 len, u64 flags)
+int __bpf_dynptr_write(const struct bpf_dynptr_kern *dst, u32 offset, void *src,
+		       u32 len, u64 flags)
 {
 	enum bpf_dynptr_type type;
 	int err;
Signature · 1 line
2.49.0
#2 lore raw

PATCH v5 2/3 bpf: implement dynptr copy kfuncs

In reply to Mykyta Yatsenko

From: Mykyta Yatsenko <[email protected]>

This patch introduces a new set of kfuncs for working with dynptrs in BPF programs, enabling reading variable-length user or kernel data into dynptr directly. To enable memory-safety, verifier allows only constant-sized reads via existing bpf_probe_read_{user|kernel} etc. kfuncs, dynptr-based kfuncs allow dynamically-sized reads without memory safety shortcomings.

The following kfuncs are introduced:
* bpf_probe_read_kernel_dynptr(): probes kernel-space data into a dynptr
* bpf_probe_read_user_dynptr(): probes user-space data into a dynptr
* bpf_probe_read_kernel_str_dynptr(): probes kernel-space string into a dynptr
* bpf_probe_read_user_str_dynptr(): probes user-space string into a dynptr
* bpf_copy_from_user_dynptr(): sleepable, copies user-space data into a dynptr for the current task
* bpf_copy_from_user_str_dynptr(): sleepable, copies user-space string into a dynptr for the current task
* bpf_copy_from_user_task_dynptr(): sleepable, copies user-space data of the task into a dynptr
* bpf_copy_from_user_task_str_dynptr(): sleepable, copies user-space string of the task into a dynptr

The implementation is built on two generic functions:
* __bpf_dynptr_copy
* __bpf_dynptr_copy_str
These functions take function pointers as arguments, enabling the copying of data from various sources, including both kernel and user space.
Use __always_inline for generic functions and callbacks to make sure the compiler doesn't generate indirect calls into callbacks, which is more expensive, especially on some kernel configurations. Inlining allows compiler to put direct calls into all the specific callback implementations (copy_user_data_sleepable, copy_user_data_nofault, and so on).

---
 kernel/bpf/helpers.c     |   8 ++
 kernel/trace/bpf_trace.c | 193 +++++++++++++++++++++++++++++++++++++++
 2 files changed, 201 insertions(+)
kernel/bpf/helpers.c +8 -0
index 048bd7ac1455..ea41eb016657 100644
@@ -3284,6 +3284,14 @@ BTF_ID_FLAGS(func, bpf_iter_kmem_cache_next, KF_ITER_NEXT | KF_RET_NULL | KF_SLE
 BTF_ID_FLAGS(func, bpf_iter_kmem_cache_destroy, KF_ITER_DESTROY | KF_SLEEPABLE)
 BTF_ID_FLAGS(func, bpf_local_irq_save)
 BTF_ID_FLAGS(func, bpf_local_irq_restore)
+BTF_ID_FLAGS(func, bpf_probe_read_user_dynptr)
+BTF_ID_FLAGS(func, bpf_probe_read_kernel_dynptr)
+BTF_ID_FLAGS(func, bpf_probe_read_user_str_dynptr)
+BTF_ID_FLAGS(func, bpf_probe_read_kernel_str_dynptr)
+BTF_ID_FLAGS(func, bpf_copy_from_user_dynptr, KF_SLEEPABLE)
+BTF_ID_FLAGS(func, bpf_copy_from_user_str_dynptr, KF_SLEEPABLE)
+BTF_ID_FLAGS(func, bpf_copy_from_user_task_dynptr, KF_SLEEPABLE | KF_TRUSTED_ARGS)
+BTF_ID_FLAGS(func, bpf_copy_from_user_task_str_dynptr, KF_SLEEPABLE | KF_TRUSTED_ARGS)
 BTF_KFUNCS_END(common_btf_ids)
 
 static const struct btf_kfunc_id_set common_kfunc_set = {
kernel/trace/bpf_trace.c +193 -0

203 lines not on this page · raw message

Signature · 1 line
2.49.0
#3 lore raw

Re: [PATCH bpf-next v5 2/3] bpf: implement dynptr copy kfuncs

kernel/trace/bpf_trace.c static int __init bpf_kprobe_multi_kfuncs_init(void) · line 3504

Quoted 5 lines · Mykyta Yatsenko
kernel/trace/bpf_trace.c +3 -0
+typedef int (*copy_fn_t)(void *dst, const void *src, u32 size, struct task_struct *tsk);
+
+/* The __always_inline is to make sure the compiler doesn't

moved it to new line as:
/*
* The __always_inline is to make sure the compiler doesn't

Please use the kernel coding style instead of the old networking style from now on.

Quoted 4 lines

+ * generate indirect calls into callbacks, which is expensive,
+ * on some kernel configurations. This allows compiler to put
+ * direct calls into all the specific callback implementations
+ *(copy_user_data_sleepable, copy_user_data_nofault, and so on)

Added extra space after *

Quoted 1 line

+ */

while applying.

#4 lore raw

PATCH v5 3/3 selftests/bpf: introduce tests for dynptr copy kfuncs

In reply to Mykyta Yatsenko

From: Mykyta Yatsenko <[email protected]>

Introduce selftests verifying newly-added dynptr copy kfuncs. Covering contiguous and non-contiguous memory backed dynptrs.

Disable test_probe_read_user_str_dynptr that triggers bug in strncpy_from_user_nofault. Patch to fix the issue [1].

[1] https://patchwork.kernel.org/project/linux-mm/patch/[email protected]/

---
 tools/testing/selftests/bpf/DENYLIST          |   1 +
 .../testing/selftests/bpf/prog_tests/dynptr.c |  13 +
 .../selftests/bpf/progs/dynptr_success.c      | 230 ++++++++++++++++++
 3 files changed, 244 insertions(+)
tools/testing/selftests/bpf/DENYLIST +1 -0
index f748f2c33b22..1789a61d0a9b 100644
@@ -1,5 +1,6 @@
 # TEMPORARY
 # Alphabetical order
+dynptr/test_probe_read_user_str_dynptr # disabled until https://patchwork.kernel.org/project/linux-mm/patch/[email protected]/ makes it into the bpf-next
 get_stack_raw_tp    # spams with kernel warnings until next bpf -> bpf-next merge
 stacktrace_build_id
 stacktrace_build_id_nmi
tools/testing/selftests/bpf/prog_tests/dynptr.c +13 -0
index e29cc16124c2..62e7ec775f24 100644
@@ -33,10 +33,19 @@ static struct {
 	{"test_dynptr_skb_no_buff", SETUP_SKB_PROG},
 	{"test_dynptr_skb_strcmp", SETUP_SKB_PROG},
 	{"test_dynptr_skb_tp_btf", SETUP_SKB_PROG_TP},
+	{"test_probe_read_user_dynptr", SETUP_XDP_PROG},
+	{"test_probe_read_kernel_dynptr", SETUP_XDP_PROG},
+	{"test_probe_read_user_str_dynptr", SETUP_XDP_PROG},
+	{"test_probe_read_kernel_str_dynptr", SETUP_XDP_PROG},
+	{"test_copy_from_user_dynptr", SETUP_SYSCALL_SLEEP},
+	{"test_copy_from_user_str_dynptr", SETUP_SYSCALL_SLEEP},
+	{"test_copy_from_user_task_dynptr", SETUP_SYSCALL_SLEEP},
+	{"test_copy_from_user_task_str_dynptr", SETUP_SYSCALL_SLEEP},
 };
 
 static void verify_success(const char *prog_name, enum test_setup_type setup_type)
 {
+	char user_data[384] = {[0 ... 382] = 'a', '\0'};
 	struct dynptr_success *skel;
 	struct bpf_program *prog;
 	struct bpf_link *link;
@@ -58,6 +67,10 @@ static void verify_success(const char *prog_name, enum test_setup_type setup_typ
 	if (!ASSERT_OK(err, "dynptr_success__load"))
 		goto cleanup;
 
+	skel->bss->user_ptr = user_data;
+	skel->data->test_len[0] = sizeof(user_data);
+	memcpy(skel->bss->expected_str, user_data, sizeof(user_data));
+
 	switch (setup_type) {
 	case SETUP_SYSCALL_SLEEP:
 		link = bpf_program__attach(prog);
tools/testing/selftests/bpf/progs/dynptr_success.c +230 -0

233 lines not on this page · raw message

Signature · 1 line
2.49.0
#6 lore raw

Re: [PATCH bpf-next v5 3/3] selftests/bpf: introduce tests for dynptr copy kfuncs

Quoted 5 lines · Alexei Starovoitov
Quoted 1 line · Mykyta Yatsenko

+dynptr/test_probe_read_user_str_dynptr # disabled until https://patchwork.kernel.org/project/linux-mm/patch/[email protected]/ makes it into the bpf-next

What is the status of it ?
I don't see it in any trees.

It was merged into the mm-nonmm-stable branch
of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm just yesterday.

#7 lore raw
In reply to Mykyta Yatsenko
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:

Hello:

This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:

Quoted 10 lines · On Mon, 12 May 2025 21:53:45 +0100 you wrote:

From: Mykyta Yatsenko <[email protected]>

This patch adds new kfuncs that enable reading variable-length user or kernel data directly into dynptrs.
These kfuncs provide a way to perform dynamically-sized reads while maintaining memory safety. Unlike existing bpf_probe_read_{user|kernel} APIs, which are limited to constant-sized reads, these new kfuncs allow for more flexible data access.

[...]

Here is the summary with links:
- [bpf-next,v5,1/3] helpers: make few bpf helpers public

    https://git.kernel.org/bpf/bpf-next/c/d060b6aab031
  - [bpf-next,v5,2/3] bpf: implement dynptr copy kfuncs
    https://git.kernel.org/bpf/bpf-next/c/a498ee7576de
  - [bpf-next,v5,3/3] selftests/bpf: introduce tests for dynptr copy kfuncs
    https://git.kernel.org/bpf/bpf-next/c/c61bcd29eda9

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html