Skip to contentSkip to conversation

bpf: Add support to allocate user page

RFC In review Jiri Olsa 7 participants 24 messages Oct 21 – Oct 26

Subsystems (9)
lore
Share & tools
lore

Jiri Olsa

author reviewer
#0 lore raw

cover bpf: Add support to allocate user page

hi,
we have few use cases where having the ability to allocate normal or executable user space memory would help (like to replace function in live process or change environment variables)

They also need writes to user memory to handle page faults, while the existing bpf_probe_write_user() helper disables faults during the copy.

This patchset adds the following kfuncs for sleepable uprobe programs:

- long bpf_user_page_alloc(unsigned long nr_pages) allocate lazy anonymous pages in the current process

- int bpf_user_page_exec(unsigned long addr)
make a page readable and executable, provided it was allocated by bpf_user_page_alloc()

- int bpf_probe_write_user_faultable(void __user *unsafe_dst__ign, const void *src__srcptr, u32 src__sz) copy data to user memory while allowing page faults to resolve the destination mapping.

I'm not sure it's the best way how to do this, hence the RFC post.

thanks for comments,
jirka

---
Jiri Olsa (9):
bpf: Add __srcptr kfunc argument suffix
bpf: Add uprobe faultable user copy kfunc
mm: Expose locked mprotect operation
bpf: Add kfuncs to allocate and seal user pages selftests/bpf: Add __srcptr kfunc arguments test selftests/bpf: Add user page allocation test selftests/bpf: Add uprobe user page exec test selftests/bpf: Add faultable user page writes test selftests/bpf: Add user page failures test

 Documentation/bpf/kfuncs.rst                               |  23 ++++
 include/linux/mm.h                                         |   3 +
 kernel/bpf/helpers.c                                       | 178 +++++++++++++++++++++++++++
 kernel/bpf/verifier.c                                      |  18 ++-
 mm/mprotect.c                                              |  20 ++-
 tools/testing/selftests/bpf/prog_tests/kfunc_call.c        |   2 +
 tools/testing/selftests/bpf/prog_tests/user_page.c         | 346 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 tools/testing/selftests/bpf/progs/kfunc_call_fail.c        |  10 ++
 tools/testing/selftests/bpf/progs/kfunc_call_test.c        |  12 ++
 tools/testing/selftests/bpf/progs/user_page.c              |  61 +++++++++
 tools/testing/selftests/bpf/test_kmods/bpf_testmod.c       |   5 +
 tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h |   1 +
 12 files changed, 672 insertions(+), 7 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/user_page.c
 create mode 100644 tools/testing/selftests/bpf/progs/user_page.c
author reviewer
#1 lore raw

RFC 1/9 bpf: Add __srcptr kfunc argument suffix

Generic sized pointer arguments are treated as writable memory. This prevents kfuncs that only read from a source buffer from accepting read-only BPF memory, such as const data.

Adding __srcptr to mark pointers the kfunc only reads.

Assisted-by Codex:GPT-5
---
 Documentation/bpf/kfuncs.rst | 23 +++++++++++++++++++++++
 kernel/bpf/verifier.c        | 18 ++++++++++++++++--
 2 files changed, 39 insertions(+), 2 deletions(-)
Documentation/bpf/kfuncs.rst +23 -0
index 73578a4c3b1f..6466b76fd721 100644
@@ -351,6 +351,29 @@ In this case, a program attached to the ``inode_init_security`` LSM hook
 can pass the hook's own ``xattr_count`` argument through to the kfunc,
 which claims xattr slots by writing through it.
 
+2.3.10 __srcptr Annotation
+-------------------------
+
+This annotation marks a pointer argument as a source buffer that the kfunc
+reads without writing. The verifier requires read access, so the source may
+point to read-only or read/write memory. Pair variable-size buffers with a
+following ``__sz`` or ``__szk`` argument. For a scalar pointer, the verifier
+uses the pointee's BTF size; a ``void *`` requires an explicit size.
+
+For example, the first kfunc takes a variable-size buffer, while the second
+takes a scalar source::
+
+        __bpf_kfunc void bpf_consume_data(const void *src__srcptr,
+                                          u32 src__sz)
+        {
+        ...
+        }
+
+        __bpf_kfunc void bpf_consume_scalar(const u32 *src__srcptr)
+        {
+        ...
+        }
+
 .. _BPF_kfunc_nodef:
 
 2.4 Using an existing kernel function
kernel/bpf/verifier.c +16 -2
index 5601d2056218..7dd7b172a297 100644
@@ -13205,6 +13205,12 @@ static bool is_kfunc_arg_mem_size(const struct btf *btf,
 	return btf_param_match_suffix(btf, arg, "__sz");
 }
 
+static bool is_kfunc_arg_srcptr(const struct btf *btf,
+				const struct btf_param *arg)
+{
+	return btf_param_match_suffix(btf, arg, "__srcptr");
+}
+
 static bool is_kfunc_arg_const_mem_size(const struct btf *btf,
 					const struct btf_param *arg)
 {
@@ -13966,6 +13972,14 @@ u32 btf_func_arg_align(const struct btf *btf, const struct btf_type *t)
 	return __btf_func_arg_align(btf, t, 0);
 }
 
+static int mem_prot(const struct btf *btf, const struct btf_param *arg)
+{
+	if (is_kfunc_arg_srcptr(btf, arg))
+		return MEM_RDONLY;
+	else
+		return MEM_WRITE;
+}
+
 static int
 get_kfunc_arg_type(struct bpf_verifier_env *env, struct bpf_call_arg_meta *meta,
 		   const struct btf_param *args, int arg, int nargs, u32 slot,
@@ -14137,7 +14151,7 @@ get_kfunc_arg_type(struct bpf_verifier_env *env, struct bpf_call_arg_meta *meta,
 				reg_arg_name(env, argno), btf_type_str(ref_t), ref_tname);
 			return -EINVAL;
 		}
-		arg_type = ARG_PTR_TO_MEM | MEM_WRITE;
+		arg_type = ARG_PTR_TO_MEM | mem_prot(meta->btf, &args[arg]);
 	} else if (btf_type_is_struct(ref_t))
 		/* A pointer to a struct without a size argument is classified as ARG_PTR_TO_BTF_ID */
 		arg_type = ARG_PTR_TO_BTF_ID;
@@ -14162,7 +14176,7 @@ get_kfunc_arg_type(struct bpf_verifier_env *env, struct bpf_call_arg_meta *meta,
 			return -EINVAL;
 		}
 		proto->arg_size[arg] = type_size;
-		arg_type = ARG_PTR_TO_MEM | MEM_FIXED_SIZE | MEM_WRITE;
+		arg_type = ARG_PTR_TO_MEM | MEM_FIXED_SIZE | mem_prot(meta->btf, &args[arg]);
 	}
 
 	if (is_kfunc_arg_uninit(meta->btf, &args[arg]))
Signature · 1 line
2.54.0
#2 lore raw

Re: [RFC bpf-next 1/9] bpf: Add __srcptr kfunc argument suffix

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [Medium] bpf: __srcptr ignored for scalar-struct pointers in resolve_func_arg_type() - [Low] bpf: docs…

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Medium] bpf: __srcptr ignored for scalar-struct pointers in resolve_func_arg_type()
- [Low] bpf: docs: short section underline for __srcptr in kfuncs.rst
--

commit 5dcfefa11540873e12bd9cbd04017e587c3a3f23
Author: Jiri Olsa <[email protected]>

bpf: Add __srcptr kfunc argument suffix

This patch adds the __srcptr suffix annotation for BPF kfunc pointer arguments so that the verifier checks them with MEM_RDONLY instead of MEM_WRITE. It also documents the new annotation in Documentation/bpf/kfuncs.rst.

Documentation/bpf/kfuncs.rst In this case, a program attached to the ``inode_init_security`` LSM hook · line 355

Quoted 10 lines
Documentation/bpf/kfuncs.rst +2 -0
index 73578a4c3b1fb..6466b76fd721e 100644
@@ -351,6 +351,29 @@ In this case, a program attached to the ``inode_init_security`` LSM hook
 can pass the hook's own ``xattr_count`` argument through to the kfunc,
 which claims xattr slots by writing through it.
 
+2.3.10 __srcptr Annotation
+-------------------------

[Severity: Low]
The section underline in Documentation/bpf/kfuncs.rst is 25 dashes while the heading is 26 characters long, which causes Sphinx to emit a "Title underline too short" warning when building the documentation. Should one more dash be added here?

[ ... ]

Quoted 4 lines
kernel/bpf/verifier.c +0 -0
index 5601d20562186..7dd7b172a2971 100644

[ ... ]

Quoted 10 lines
kernel/bpf/verifier.c +1 -1
@@ -14137,7 +14151,7 @@ get_kfunc_arg_type(struct bpf_verifier_env *env, struct bpf_call_arg_meta *meta,
 				reg_arg_name(env, argno), btf_type_str(ref_t), ref_tname);
 			return -EINVAL;
 		}
-		arg_type = ARG_PTR_TO_MEM | MEM_WRITE;
+		arg_type = ARG_PTR_TO_MEM | mem_prot(meta->btf, &args[arg]);
 	} else if (btf_type_is_struct(ref_t))
 		/* A pointer to a struct without a size argument is classified as ARG_PTR_TO_BTF_ID */
 		arg_type = ARG_PTR_TO_BTF_ID;

kernel/bpf/verifier.c get_kfunc_arg_type(struct bpf_verifier_env *env, struct bpf_call_arg_meta *meta, · line 14180

Quoted 6 lines
kernel/bpf/verifier.c +1 -1
@@ -14162,7 +14176,7 @@ get_kfunc_arg_type(struct bpf_verifier_env *env, struct bpf_call_arg_meta *meta,
 			return -EINVAL;
 		}
 		proto->arg_size[arg] = type_size;
-		arg_type = ARG_PTR_TO_MEM | MEM_FIXED_SIZE | MEM_WRITE;
+		arg_type = ARG_PTR_TO_MEM | MEM_FIXED_SIZE | mem_prot(meta->btf, &args[arg]);
 	}

[Severity: Medium]
Should resolve_func_arg_type() in kernel/bpf/verifier.c also use mem_prot(meta->btf, &args[arg]) instead of hardcoding MEM_WRITE?

In get_kfunc_arg_type(), a pointer to a struct or union without a following __sz or __szk argument matches btf_type_is_struct(ref_t) and is classified as ARG_PTR_TO_BTF_ID, bypassing the final else branch above.

Later, when a BPF program passes a non-BTF-ID memory pointer (such as read-only map values from .rodata, PTR_TO_MEM | MEM_RDONLY, map keys, or stack memory) to a kfunc parameter pointing to a fixed-size struct or union composed of scalars, check_func_arg() -> resolve_func_arg_type() converts the argument type to fixed-size memory, but still unconditionally sets MEM_WRITE:

kernel/bpf/verifier.c:resolve_func_arg_type() {
	...
	if (!btf_type_is_scalar_struct(env, btf, ref_t))
		return 0;

	resolve_ret = btf_resolve_size(btf, ref_t, arg_size);
	if (IS_ERR(resolve_ret)) {
		...
	}
	*arg_type = ARG_PTR_TO_MEM | MEM_FIXED_SIZE | MEM_WRITE |
		    (*arg_type & (PTR_MAYBE_NULL | MEM_UNINIT));

	return 0;
}

Because MEM_WRITE remains set for scalar-struct pointers annotated with __srcptr, won't the verifier still reject read-only memory arguments and clobber spilled stack slots?

Quoted 2 lines
kernel/bpf/verifier.c +0 -0
 	if (is_kfunc_arg_uninit(meta->btf, &args[arg]))
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
author reviewer
#3 lore raw

RFC 2/9 bpf: Add uprobe faultable user copy kfunc

In reply to Jiri Olsa

The bpf_probe_write_user() helper uses a no-fault copy. Adding kfunc that can fault in the destination pages while copying to user memory.

Restricting it to sleepable uprobes and to LOCKDOWN_BPF_WRITE_USER.

Assisted-by Codex:GPT-6
---
 kernel/bpf/helpers.c | 57 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 57 insertions(+)
kernel/bpf/helpers.c +57 -0
index 648abe1ba63a..4d9e9423826a 100644
@@ -29,6 +29,7 @@
 #include <linux/task_work.h>
 #include <linux/irq_work.h>
 #include <linux/buildid.h>
+#include <asm/tlb.h>
 
 #include "../../lib/kstrtox.h"
 
@@ -3738,6 +3739,36 @@ __bpf_kfunc int bpf_copy_from_user_mm(void *dst, u32 dst__sz,
 	return ret;
 }
 
+/**
+ * bpf_probe_write_user_faultable() - Copy data to userspace with page faults
+ * enabled
+ * @unsafe_dst__ign: Destination address in the current task's address space.
+ * @src__srcptr:     Source buffer in BPF-accessible memory.
+ * @src__sz:         Number of bytes to copy.
+ *
+ * Copies data to userspace, allowing page faults to resolve the destination
+ * mapping. A failed copy may have written a prefix of the source buffer.
+ *
+ * Return: 0 on success, -EFAULT if the copy is incomplete, or -EPERM if the
+ * current task cannot safely access userspace.
+ */
+__bpf_kfunc int bpf_probe_write_user_faultable(void __user *unsafe_dst__ign,
+					      const void *src__srcptr,
+					      u32 src__sz)
+{
+	if (unlikely(in_interrupt() ||
+		     current->flags & (PF_KTHREAD | PF_EXITING)))
+		return -EPERM;
+	if (unlikely(!nmi_uaccess_okay()))
+		return -EPERM;
+	if (unlikely(!current->mm))
+		return -EFAULT;
+	if (copy_to_user(unsafe_dst__ign, src__srcptr, src__sz))
+		return -EFAULT;
+
+	return 0;
+}
+
 /**
  * bpf_copy_from_user_mm_str() - Copy a string from an address space
  * @dst:             Destination address, in kernel space. This buffer must be
@@ -5139,6 +5170,31 @@ static const struct btf_kfunc_id_set generic_kfunc_set = {
 	.set   = &generic_btf_ids,
 };
 
+BTF_KFUNCS_START(user_mem_btf_ids)
+BTF_ID_FLAGS(func, bpf_probe_write_user_faultable, KF_SLEEPABLE | KF_PERFMON)
+BTF_KFUNCS_END(user_mem_btf_ids)
+
+enum user_mem_kfunc_id {
+	KF_bpf_probe_write_user_faultable,
+};
+
+BTF_ID_LIST_SINGLE(bpf_probe_write_user_faultable_btf_id, func,
+		   bpf_probe_write_user_faultable)
+
+static int user_mem_kfunc_filter(const struct bpf_prog *prog, u32 kfunc_id)
+{
+	if (kfunc_id == bpf_probe_write_user_faultable_btf_id[0])
+		return security_locked_down(LOCKDOWN_BPF_WRITE_USER);
+
+	return 0;
+}
+
+static const struct btf_kfunc_id_set user_mem_kfunc_set = {
+	.owner = THIS_MODULE,
+	.set = &user_mem_btf_ids,
+	.filter = user_mem_kfunc_filter,
+};
+
 BTF_ID_LIST(generic_dtor_ids)
 BTF_ID(struct, task_struct)
 BTF_ID(func, bpf_task_release_dtor)
@@ -5266,6 +5322,7 @@ static int __init kfunc_init(void)
 	};
 
 	ret = register_btf_kfunc_id_set(BPF_PROG_TYPE_TRACING, &generic_kfunc_set);
+	ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_KPROBE, &user_mem_kfunc_set);
 	ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_SCHED_CLS, &generic_kfunc_set);
 	ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_XDP, &generic_kfunc_set);
 	ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_STRUCT_OPS, &generic_kfunc_set);
Signature · 1 line
2.54.0
#4 lore raw

Re: [RFC bpf-next 2/9] bpf: Add uprobe faultable user copy kfunc

Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider: - [Critical] bpf: missing CAP_SYS_ADMIN check in user_mem_kfunc_filter() - [High] bpf: hardened usercopy…

Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:
- [Critical] bpf: missing CAP_SYS_ADMIN check in user_mem_kfunc_filter()
- [High] bpf: hardened usercopy BUG() in bpf_probe_write_user_faultable()
- [Medium] bpf: missing prog->sleepable check in user_mem_kfunc_filter()
- [Low] bpf: unused enum user_mem_kfunc_id in kernel/bpf/helpers.c
--

commit bbd6dece6abea0ce7dfa98ce448c7aae57de64d1
Author: Jiri Olsa <[email protected]>

bpf: Add uprobe faultable user copy kfunc

This patch adds the bpf_probe_write_user_faultable() sleepable kfunc to copy data to user memory with page faults enabled. It registers the kfunc for BPF_PROG_TYPE_KPROBE programs and filters access with LOCKDOWN_BPF_WRITE_USER.

Quoted 4 lines
kernel/bpf/helpers.c +0 -0
index 648abe1ba63ae..4d9e9423826a9 100644

[ ... ]

Quoted 20 lines
kernel/bpf/helpers.c +16 -0
@@ -3738,6 +3739,36 @@ __bpf_kfunc int bpf_copy_from_user_mm(void *dst, u32 dst__sz,
 	return ret;
 }
 
+/**
+ * bpf_probe_write_user_faultable() - Copy data to userspace with page faults
+ * enabled
+ * @unsafe_dst__ign: Destination address in the current task's address space.
+ * @src__srcptr:     Source buffer in BPF-accessible memory.
+ * @src__sz:         Number of bytes to copy.
+ *
+ * Copies data to userspace, allowing page faults to resolve the destination
+ * mapping. A failed copy may have written a prefix of the source buffer.
+ *
+ * Return: 0 on success, -EFAULT if the copy is incomplete, or -EPERM if the
+ * current task cannot safely access userspace.
+ */
+__bpf_kfunc int bpf_probe_write_user_faultable(void __user *unsafe_dst__ign,
+					      const void *src__srcptr,
+					      u32 src__sz)

kernel/bpf/helpers.c __bpf_kfunc int bpf_copy_from_user_mm(void *dst, u32 dst__sz, · line 3767

Quoted 10 lines
kernel/bpf/helpers.c +10 -0
+{
+	if (unlikely(in_interrupt() ||
+		     current->flags & (PF_KTHREAD | PF_EXITING)))
+		return -EPERM;
+	if (unlikely(!nmi_uaccess_okay()))
+		return -EPERM;
+	if (unlikely(!current->mm))
+		return -EFAULT;
+	if (copy_to_user(unsafe_dst__ign, src__srcptr, src__sz))
+		return -EFAULT;

[Severity: High]
Can calling copy_to_user() directly on src__srcptr trigger a hardened usercopy bug when a BPF program passes a trusted BTF object pointer?

In kernel/bpf/verifier.c, get_kfunc_arg_type() classifies src__srcptr paired with src__sz as ARG_PTR_TO_MEM | MEM_RDONLY. During verification, check_reg_type() permits mem_types, which includes:

kernel/bpf/verifier.c:
PTR_TO_BTF_ID | PTR_TRUSTED,

and check_helper_mem_access() allows BPF_READ access on scalar fields of trusted BTF objects (such as &task->pid or task->comm from bpf_get_current_task_btf()).

At runtime, copy_to_user() calls check_copy_size(src__srcptr, src__sz, true) -> __check_object_size() -> __check_heap_object() in mm/slub.c:

mm/slub.c:__check_heap_object() {
	...
	usercopy_abort("SLUB object", s->name, to_user, offset, n);
}

When src__srcptr points to a field in a dedicated slab cache outside its whitelisted usercopy region (for example task_struct_cachep, mm_cachep, files_cachep, or fs_cachep), wouldn't __check_heap_object() call usercopy_abort() and panic the kernel when CONFIG_HARDENED_USERCOPY is enabled?

Quoted 8 lines
kernel/bpf/helpers.c +4 -0
+
+	return 0;
+}
+
 /**
  * bpf_copy_from_user_mm_str() - Copy a string from an address space
  * @dst:             Destination address, in kernel space. This buffer must be

kernel/bpf/helpers.c static const struct btf_kfunc_id_set generic_kfunc_set = { · line 5179

Quoted 10 lines
kernel/bpf/helpers.c +7 -0
@@ -5139,6 +5170,31 @@ static const struct btf_kfunc_id_set generic_kfunc_set = {
 	.set   = &generic_btf_ids,
 };
 
+BTF_KFUNCS_START(user_mem_btf_ids)
+BTF_ID_FLAGS(func, bpf_probe_write_user_faultable, KF_SLEEPABLE | KF_PERFMON)
+BTF_KFUNCS_END(user_mem_btf_ids)
+
+enum user_mem_kfunc_id {
+	KF_bpf_probe_write_user_faultable,
+};

[Severity: Low]
Is enum user_mem_kfunc_id left over from an earlier version of the patch? Since user_mem_kfunc_filter() compares kfunc_id against bpf_probe_write_user_faultable_btf_id[0], neither enum user_mem_kfunc_id nor KF_bpf_probe_write_user_faultable is used anywhere in kernel/bpf/helpers.c.

kernel/bpf/helpers.c static const struct btf_kfunc_id_set generic_kfunc_set = { · line 5187

Quoted 8 lines
kernel/bpf/helpers.c +8 -0
+
+BTF_ID_LIST_SINGLE(bpf_probe_write_user_faultable_btf_id, func,
+		   bpf_probe_write_user_faultable)
+
+static int user_mem_kfunc_filter(const struct bpf_prog *prog, u32 kfunc_id)
+{
+	if (kfunc_id == bpf_probe_write_user_faultable_btf_id[0])
+		return security_locked_down(LOCKDOWN_BPF_WRITE_USER);

[Severity: Critical]
Should user_mem_kfunc_filter() also verify that the program has CAP_SYS_ADMIN via bpf_token_capable(prog->aux->token, CAP_SYS_ADMIN)?

In user_mem_btf_ids, bpf_probe_write_user_faultable is marked with KF_PERFMON, which in check_kfunc_call() in kernel/bpf/verifier.c only checks env->allow_ptr_leaks (satisfied by CAP_PERFMON). Meanwhile, for the existing bpf_probe_write_user() helper, bpf_tracing_func_proto() in kernel/trace/bpf_trace.c enforces CAP_SYS_ADMIN in addition to lockdown:

kernel/trace/bpf_trace.c:bpf_tracing_func_proto() {
	...
	if (!bpf_token_capable(prog->aux->token, CAP_SYS_ADMIN))
		return NULL;

	switch (func_id) {
	case BPF_FUNC_probe_write_user:
		return security_locked_down(LOCKDOWN_BPF_WRITE_USER) < 0 ?
		       NULL : &bpf_probe_write_user_proto;
	...
}

And include/uapi/linux/capability.h documents this boundary under CAP_BPF:

* CAP_SYS_ADMIN is required to use bpf_probe_write_user.

Without a bpf_token_capable(prog->aux->token, CAP_SYS_ADMIN) check in user_mem_kfunc_filter(), couldn't a process or container granted only CAP_BPF and CAP_PERFMON attach a sleepable uprobe or uprobe_multi program to a shared library and overwrite memory in privileged processes?

kernel/bpf/helpers.c __bpf_kfunc int bpf_copy_from_user_mm(void *dst, u32 dst__sz, · line 3770

Quoted 3 lines
kernel/bpf/helpers.c +3 -0
+
+	return 0;
+}

[Severity: Medium]
Does user_mem_kfunc_filter() also need to check prog->sleepable to restrict bpf_probe_write_user_faultable() to sleepable uprobes as described in the commit message?

Because BPF_PROG_TYPE_KPROBE is shared by kernel kprobes and uprobes, the uprobe-only restriction for sleepable programs is enforced at attach time by checking prog->sleepable in __perf_event_set_bpf_prog() and bpf_kprobe_multi_link_attach().

In check_kfunc_call() in kernel/bpf/verifier.c, KF_SLEEPABLE checks in_sleepable_context(env) rather than prog->sleepable. When a non-sleepable BPF_PROG_TYPE_KPROBE program (prog->sleepable == false) registers a callback via bpf_task_work_schedule_signal(), bpf_task_work_schedule_resume(), or bpf_wq_set_callback(), push_callback_call() marks the callback state as sleepable via is_async_cb_sleepable():

kernel/bpf/verifier.c:is_async_cb_sleepable() {
	/* bpf_wq and bpf_task_work callbacks are always sleepable. */
	if (bpf_pseudo_kfunc_call(insn) && insn->off == 0 &&
	    (is_bpf_wq_set_callback_kfunc(insn->imm) || is_task_work_add_kfunc(insn->imm)))
		return true;
	return false;
}

Without checking !prog->sleepable in user_mem_kfunc_filter(), wouldn't a non-sleepable kernel kprobe, kretprobe, or kprobe_multi program be able to call bpf_probe_write_user_faultable() from a bpf_task_work callback?

Quoted 6 lines
kernel/bpf/helpers.c +6 -0
+
+static const struct btf_kfunc_id_set user_mem_kfunc_set = {
+	.owner = THIS_MODULE,
+	.set = &user_mem_btf_ids,
+	.filter = user_mem_kfunc_filter,
+};

[ ... ]

Quoted 5 lines
kernel/bpf/helpers.c +1 -0
@@ -5266,6 +5322,7 @@ static int __init kfunc_init(void)
 	};
 
 	ret = register_btf_kfunc_id_set(BPF_PROG_TYPE_TRACING, &generic_kfunc_set);
+	ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_KPROBE, &user_mem_kfunc_set);

[ ... ]

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
#5 lore raw

Re: [RFC bpf-next 2/9] bpf: Add uprobe faultable user copy kfunc

In reply to Jiri Olsa
TL;DR: - New write-to-user kfunc may let CAP_BPF+CAP_PERFMON loaders write user memory, bypassing the helper's CAP_SYS_ADMIN requirement - Unused enum (nit)
Quoted 1 line

bpf: Add uprobe faultable user copy kfunc

TL;DR:
- New write-to-user kfunc may let CAP_BPF+CAP_PERFMON loaders write user memory, bypassing the helper's CAP_SYS_ADMIN requirement
- Unused enum (nit)

Quoted 3 lines
kernel/bpf/helpers.c +0 -0

[ ... ]

kernel/bpf/helpers.c static const struct btf_kfunc_id_set generic_kfunc_set = { · line 5175

Quoted 7 lines
kernel/bpf/helpers.c +3 -0
@@ -5139,6 +5170,31 @@ static const struct btf_kfunc_id_set generic_kfunc_set = {
 	.set   = &generic_btf_ids,
 };
 
+BTF_KFUNCS_START(user_mem_btf_ids)
+BTF_ID_FLAGS(func, bpf_probe_write_user_faultable, KF_SLEEPABLE | KF_PERFMON)
+BTF_KFUNCS_END(user_mem_btf_ids)

The new kfunc does the same job as bpf_probe_write_user(), but it is gated only by KF_PERFMON. The verifier checks that flag as env->allow_ptr_leaks, which is bpf_token_capable(token, CAP_PERFMON), and loading a BPF_PROG_TYPE_KPROBE program already requires CAP_PERFMON (see is_perfmon_prog_type() in syscall.c). For this program type KF_PERFMON doesn't seem to add any restriction.

The existing helper is gated on CAP_SYS_ADMIN in bpf_tracing_func_proto():

    if (!bpf_token_capable(prog->aux->token, CAP_SYS_ADMIN))
        return NULL;
    switch (func_id) {
    case BPF_FUNC_probe_write_user:
        return security_locked_down(LOCKDOWN_BPF_WRITE_USER) < 0 ?
               NULL : &bpf_probe_write_user_proto;

and include/uapi/linux/capability.h documents "CAP_SYS_ADMIN is required to use bpf_probe_write_user."

With this patch, can a loader holding only CAP_BPF and CAP_PERFMON write to user memory through the kfunc even though the helper refuses it?

The commit message carries over the LOCKDOWN_BPF_WRITE_USER restriction from the helper but says nothing about the capability requirement. Should user_mem_kfunc_filter() also require bpf_token_capable(prog->aux->token, CAP_SYS_ADMIN) for this kfunc? If CAP_PERFMON is meant to be enough, could the changelog and capability.h explain why?

kernel/bpf/helpers.c static const struct btf_kfunc_id_set generic_kfunc_set = { · line 5180

Quoted 5 lines
kernel/bpf/helpers.c +5 -0
+
+enum user_mem_kfunc_id {
+	KF_bpf_probe_write_user_faultable,
+};
+

This isn't a bug, but enum user_mem_kfunc_id and its enumerator KF_bpf_probe_write_user_faultable don't seem to be referenced anywhere, in this commit or later in the series. user_mem_kfunc_filter() compares against bpf_probe_write_user_faultable_btf_id[0] from the BTF_ID_LIST_SINGLE below instead.

The similar pattern in verifier.c (enum special_kfunc_type) works because it indexes a BTF_ID_LIST, and there is no such list here. Can the enum be dropped?

kernel/bpf/helpers.c static const struct btf_kfunc_id_set generic_kfunc_set = { · line 5190

Quoted 10 lines
kernel/bpf/helpers.c +10 -0
+BTF_ID_LIST_SINGLE(bpf_probe_write_user_faultable_btf_id, func,
+		   bpf_probe_write_user_faultable)
+
+static int user_mem_kfunc_filter(const struct bpf_prog *prog, u32 kfunc_id)
+{
+	if (kfunc_id == bpf_probe_write_user_faultable_btf_id[0])
+		return security_locked_down(LOCKDOWN_BPF_WRITE_USER);
+
+	return 0;
+}

[ ... ]

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/39668036298

author reviewer
#6 lore raw

RFC 3/9 mm: Expose locked mprotect operation

In reply to Jiri Olsa

The bpf_user_page_exec() in following patch needs to change page protections while holding current->mm's mmap write lock.

Exposing the do_mprotect_pkey_locked helper for that.

Assisted-by Codex:GPT-6
---
 include/linux/mm.h |  3 +++
 mm/mprotect.c      | 20 +++++++++++++++-----
 2 files changed, 18 insertions(+), 5 deletions(-)
include/linux/mm.h +3 -0
index 63f40e615754..47e853093601 100644
@@ -3435,6 +3435,9 @@ extern long change_protection(struct mmu_gather *tlb,
 extern int mprotect_fixup(struct vma_iterator *vmi, struct mmu_gather *tlb,
 	  struct vm_area_struct *vma, struct vm_area_struct **pprev,
 	  unsigned long start, unsigned long end, vm_flags_t newflags);
+/* Caller must hold current->mm's mmap write lock. */
+int do_mprotect_pkey_locked(unsigned long start, size_t len,
+			    unsigned long prot, int pkey);
 
 /*
  * doesn't attempt to fault and will return short.
mm/mprotect.c +15 -5
index 2888ee638d87..8d800830d3e1 100644
@@ -868,8 +868,8 @@ mprotect_fixup(struct vma_iterator *vmi, struct mmu_gather *tlb,
 /*
  * pkey==-1 when doing a legacy mprotect()
  */
-static int do_mprotect_pkey(unsigned long start, size_t len,
-		unsigned long prot, int pkey)
+int do_mprotect_pkey_locked(unsigned long start, size_t len,
+			    unsigned long prot, int pkey)
 {
 	unsigned long nstart, end, tmp, reqprot;
 	struct vm_area_struct *vma, *prev;
@@ -899,8 +899,7 @@ static int do_mprotect_pkey(unsigned long start, size_t len,
 
 	reqprot = prot;
 
-	if (mmap_write_lock_killable(current->mm))
-		return -EINTR;
+	mmap_assert_write_locked(current->mm);
 
 	/*
 	 * If userspace did not allocate the pkey, do not let
@@ -1013,10 +1012,21 @@ static int do_mprotect_pkey(unsigned long start, size_t len,
 		error = -ENOMEM;
 
 out:
-	mmap_write_unlock(current->mm);
 	return error;
 }
 
+static int do_mprotect_pkey(unsigned long start, size_t len,
+			    unsigned long prot, int pkey)
+{
+	int ret;
+
+	if (mmap_write_lock_killable(current->mm))
+		return -EINTR;
+	ret = do_mprotect_pkey_locked(start, len, prot, pkey);
+	mmap_write_unlock(current->mm);
+	return ret;
+}
+
 SYSCALL_DEFINE3(mprotect, unsigned long, start, size_t, len,
 		unsigned long, prot)
 {
Signature · 1 line
2.54.0
#7 lore raw

Re: [RFC bpf-next 3/9] mm: Expose locked mprotect operation

TL;DR: do_mprotect_pkey_locked() has no !CONFIG_MMU stub, so once the next patch calls it, CONFIG_MMU=n with CONFIG_BPF_SYSCALL=y kernels may fail to link.
Quoted 1 line

mm: Expose locked mprotect operation

TL;DR: do_mprotect_pkey_locked() has no !CONFIG_MMU stub, so once the next patch calls it, CONFIG_MMU=n with CONFIG_BPF_SYSCALL=y kernels may fail to link.

include/linux/mm.h extern long change_protection(struct mmu_gather *tlb, · line 3440

Quoted 11 lines
include/linux/mm.h +3 -0
index 63f40e615754..47e853093601 100644
@@ -3435,6 +3435,9 @@ extern long change_protection(struct mmu_gather *tlb,
 extern int mprotect_fixup(struct vma_iterator *vmi, struct mmu_gather *tlb,
 	  struct vm_area_struct *vma, struct vm_area_struct **pprev,
 	  unsigned long start, unsigned long end, vm_flags_t newflags);
+/* Caller must hold current->mm's mmap write lock. */
+int do_mprotect_pkey_locked(unsigned long start, size_t len,
+			    unsigned long prot, int pkey);

Can this declaration lead to an undefined reference on kernels built with CONFIG_MMU=n and CONFIG_BPF_SYSCALL=y?

The prototype is unconditional and there is no !CONFIG_MMU stub, but the only definition is in mm/mprotect.c, which is built only for MMU kernels:

mm/Makefile:

    mmu-$(CONFIG_MMU)	:= ... mprotect.o ...

The next patch in the series, f62943356e31 ("bpf: Add kfuncs to allocate and seal user pages"), calls the helper from bpf_user_page_exec() in kernel/bpf/helpers.c with no CONFIG_MMU guard:

    err = do_mprotect_pkey_locked(addr, PAGE_SIZE, PROT_READ | PROT_EXEC, -1);

helpers.o is built for any CONFIG_BPF_SYSCALL=y kernel, and BPF_SYSCALL has no MMU dependency in kernel/bpf/Kconfig. Builds without an MMU do appear to be supported for bpf: arena.o is explicitly limited to
"ifeq ($(CONFIG_MMU)$(CONFIG_64BIT),yy)", and ced6a0993e2f fixed a !MMU/32-bit bpf build break.

So once f62943356e31 is applied, would a CONFIG_MMU=n, CONFIG_BPF_SYSCALL=y kernel fail to link? This patch builds on its own, and nothing up to the end of the series (1b0c19c85c7f) appears to add a fix.

Would it make sense to either declare the helper under "#ifdef CONFIG_MMU" with a static inline stub that returns an error, or add a CONFIG_MMU guard around the kfunc and its BTF_ID_FLAGS entry in the bpf patch?

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/39668036298

#8 lore raw

Re: [RFC bpf-next 3/9] mm: Expose locked mprotect operation

In reply to Jiri Olsa
Quoted 4 lines · Jiri Olsa

The bpf_user_page_exec() in following patch needs to change page protections while holding current->mm's mmap write lock.

Exposing the do_mprotect_pkey_locked helper for that.

Err, we'll need some more justification than this for exposing internal mm logic to the rest of the kernel please.

Note that you're already allowing any caller to bypass CONFIG_ARCH_HAS_PKEYS here which is immediately a broken thing.

Quoted 3 lines
Assisted-by Codex:GPT-6

It's a no to this in its current form.

Quoted 68 lines
---
 include/linux/mm.h |  3 +++
 mm/mprotect.c      | 20 +++++++++++++++-----
 2 files changed, 18 insertions(+), 5 deletions(-)
include/linux/mm.h +3 -0
index 63f40e615754..47e853093601 100644
@@ -3435,6 +3435,9 @@ extern long change_protection(struct mmu_gather *tlb,
 extern int mprotect_fixup(struct vma_iterator *vmi, struct mmu_gather *tlb,
 	  struct vm_area_struct *vma, struct vm_area_struct **pprev,
 	  unsigned long start, unsigned long end, vm_flags_t newflags);
+/* Caller must hold current->mm's mmap write lock. */
+int do_mprotect_pkey_locked(unsigned long start, size_t len,
+			    unsigned long prot, int pkey);
 
 /*
  * doesn't attempt to fault and will return short.
mm/mprotect.c +15 -6
index 2888ee638d87..8d800830d3e1 100644
@@ -868,8 +868,8 @@ mprotect_fixup(struct vma_iterator *vmi, struct mmu_gather *tlb,
 /*
  * pkey==-1 when doing a legacy mprotect()
  */
-static int do_mprotect_pkey(unsigned long start, size_t len,
-		unsigned long prot, int pkey)
+int do_mprotect_pkey_locked(unsigned long start, size_t len,
+			    unsigned long prot, int pkey)
 {
 	unsigned long nstart, end, tmp, reqprot;
 	struct vm_area_struct *vma, *prev;
@@ -899,8 +899,7 @@ static int do_mprotect_pkey(unsigned long start, size_t len,
 
 	reqprot = prot;
 
-	if (mmap_write_lock_killable(current->mm))
-		return -EINTR;
+	mmap_assert_write_locked(current->mm);
 
 	/*
 	 * If userspace did not allocate the pkey, do not let
@@ -1013,10 +1012,21 @@ static int do_mprotect_pkey(unsigned long start, size_t len,
 		error = -ENOMEM;
 
 out:
-	mmap_write_unlock(current->mm);
 	return error;
 }
 
+static int do_mprotect_pkey(unsigned long start, size_t len,
+			    unsigned long prot, int pkey)
+{
+	int ret;
+
+	if (mmap_write_lock_killable(current->mm))
+		return -EINTR;
+	ret = do_mprotect_pkey_locked(start, len, prot, pkey);
+	mmap_write_unlock(current->mm);
+	return ret;
+}
+
 SYSCALL_DEFINE3(mprotect, unsigned long, start, size_t, len,
 		unsigned long, prot)
 {
--
Quoted 2 lines

2.54.0

--
Cheers, Lorenzo

#9 lore raw

Re: [RFC bpf-next 3/9] mm: Expose locked mprotect operation

Quoted 9 lines · Lorenzo Stoakes (ARM)
Quoted 4 lines · Jiri Olsa

The bpf_user_page_exec() in following patch needs to change page protections while holding current->mm's mmap write lock.

Exposing the do_mprotect_pkey_locked helper for that.

Err, we'll need some more justification than this for exposing internal mm logic to the rest of the kernel please.

Quoted 8 lines

Note that you're already allowing any caller to bypass CONFIG_ARCH_HAS_PKEYS here which is immediately a broken thing.

Quoted 3 lines
Assisted-by Codex:GPT-6

It's a no to this in its current form.

yes I suspected it'd be a problem, but as you mentioned in the other reply, Alexei's proposal takes care of that

thanks,
jirka

Quoted 74 lines
Quoted 69 lines
---
 include/linux/mm.h |  3 +++
 mm/mprotect.c      | 20 +++++++++++++++-----
 2 files changed, 18 insertions(+), 5 deletions(-)
include/linux/mm.h +3 -0
index 63f40e615754..47e853093601 100644
@@ -3435,6 +3435,9 @@ extern long change_protection(struct mmu_gather *tlb,
 extern int mprotect_fixup(struct vma_iterator *vmi, struct mmu_gather *tlb,
 	  struct vm_area_struct *vma, struct vm_area_struct **pprev,
 	  unsigned long start, unsigned long end, vm_flags_t newflags);
+/* Caller must hold current->mm's mmap write lock. */
+int do_mprotect_pkey_locked(unsigned long start, size_t len,
+			    unsigned long prot, int pkey);
 
 /*
  * doesn't attempt to fault and will return short.
mm/mprotect.c +15 -6
index 2888ee638d87..8d800830d3e1 100644
@@ -868,8 +868,8 @@ mprotect_fixup(struct vma_iterator *vmi, struct mmu_gather *tlb,
 /*
  * pkey==-1 when doing a legacy mprotect()
  */
-static int do_mprotect_pkey(unsigned long start, size_t len,
-		unsigned long prot, int pkey)
+int do_mprotect_pkey_locked(unsigned long start, size_t len,
+			    unsigned long prot, int pkey)
 {
 	unsigned long nstart, end, tmp, reqprot;
 	struct vm_area_struct *vma, *prev;
@@ -899,8 +899,7 @@ static int do_mprotect_pkey(unsigned long start, size_t len,
 
 	reqprot = prot;
 
-	if (mmap_write_lock_killable(current->mm))
-		return -EINTR;
+	mmap_assert_write_locked(current->mm);
 
 	/*
 	 * If userspace did not allocate the pkey, do not let
@@ -1013,10 +1012,21 @@ static int do_mprotect_pkey(unsigned long start, size_t len,
 		error = -ENOMEM;
 
 out:
-	mmap_write_unlock(current->mm);
 	return error;
 }
 
+static int do_mprotect_pkey(unsigned long start, size_t len,
+			    unsigned long prot, int pkey)
+{
+	int ret;
+
+	if (mmap_write_lock_killable(current->mm))
+		return -EINTR;
+	ret = do_mprotect_pkey_locked(start, len, prot, pkey);
+	mmap_write_unlock(current->mm);
+	return ret;
+}
+
 SYSCALL_DEFINE3(mprotect, unsigned long, start, size_t, len,
 		unsigned long, prot)
 {
--
Quoted 2 lines

2.54.0

--
Cheers, Lorenzo

author reviewer
#10 lore raw

RFC 4/9 bpf: Add kfuncs to allocate and seal user pages

In reply to Jiri Olsa

Adding sleepable BPF kfuncs for allocating anonymous pages in the current process and changing an allocated page to read/execute permissions. The mapping remains in userspace until it is unmapped or the process exits.

It's allowed only for sleepable kprobe type, which technicaly means only sleepable uprobe.

Assisted-by Codex:GPT-6
---
 kernel/bpf/helpers.c | 127 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 127 insertions(+)
kernel/bpf/helpers.c +127 -0
index 4d9e9423826a..246f4969a144 100644
@@ -29,6 +29,10 @@
 #include <linux/task_work.h>
 #include <linux/irq_work.h>
 #include <linux/buildid.h>
+#include <linux/mm.h>
+#include <linux/mman.h>
+#include <linux/personality.h>
+#include <asm/cacheflush.h>
 #include <asm/tlb.h>
 
 #include "../../lib/kstrtox.h"
@@ -3739,6 +3743,127 @@ __bpf_kfunc int bpf_copy_from_user_mm(void *dst, u32 dst__sz,
 	return ret;
 }
 
+/* vm_private_data token for the unmergeable anonymous mapping below. */
+static char bpf_user_page_marker;
+
+/**
+ * bpf_user_page_alloc() - Allocate lazy anonymous pages in the current mm
+ * @nr_pages: Number of pages to allocate
+ *
+ * Creates a read/write anonymous mapping in the current task's address space.
+ * The mapping is kept non-executable even for tasks with READ_IMPLIES_EXEC,
+ * to avoid creating a writable executable mapping. The physical page is
+ * allocated on its first access. The mapping remains until userspace unmaps
+ * it or the process exits.
+ *
+ * Return: The userspace address on success, or a negative error code.
+ */
+__bpf_kfunc long bpf_user_page_alloc(unsigned long nr_pages)
+{
+	const unsigned long prot = PROT_READ | PROT_WRITE;
+	const unsigned long flags = MAP_PRIVATE | MAP_ANONYMOUS;
+	struct mm_struct *mm = current->mm;
+	unsigned long addr, populate, len;
+	struct vm_area_struct *vma;
+	int err;
+
+	if (unlikely(in_interrupt() || current->flags & (PF_KTHREAD | PF_EXITING)))
+		return -EPERM;
+	if (unlikely(!nmi_uaccess_okay()))
+		return -EPERM;
+	if (unlikely(!mm))
+		return -EFAULT;
+	if (unlikely(!nr_pages || nr_pages > (ULONG_MAX >> PAGE_SHIFT)))
+		return -EINVAL;
+
+	len = nr_pages << PAGE_SHIFT;
+	err = security_mmap_file(NULL, prot, flags);
+	if (err)
+		return err;
+
+	if (mmap_write_lock_killable(mm))
+		return -EINTR;
+
+	addr = do_mmap(NULL, 0, len, prot, flags,
+		       mk_vma_flags(VMA_DONTEXPAND_BIT, VMA_DONTCOPY_BIT, VMA_DONTDUMP_BIT),
+		       0, &populate, NULL);
+	if (!IS_ERR_VALUE(addr)) {
+		/* VM_DONTEXPAND prevents merging this mapping into another VMA. */
+		vma = find_vma(mm, addr);
+		vma_start_write(vma);
+		if (current->personality & READ_IMPLIES_EXEC) {
+			vm_flags_clear(vma, VM_EXEC);
+			vma_set_page_prot(vma);
+		}
+		vma->vm_private_data = &bpf_user_page_marker;
+	}
+
+	mmap_write_unlock(mm);
+
+	if (!IS_ERR_VALUE(addr) && populate)
+		mm_populate(addr, populate);
+	return addr;
+}
+
+/**
+ * bpf_user_page_exec() - Make a BPF-allocated user page read/execute
+ * @addr: Address of the page to change
+ *
+ * Changes a writable page created by bpf_user_page_alloc() to read/execute.
+ * The page cannot be made writable again after this call.
+ *
+ * Return: 0 on success, or a negative error code.
+ */
+__bpf_kfunc int bpf_user_page_exec(unsigned long addr)
+{
+	struct mm_struct *mm = current->mm;
+	struct vm_area_struct *vma;
+	struct page *page;
+	int err = -EINVAL;
+
+	if (unlikely(in_interrupt() ||
+		     current->flags & (PF_KTHREAD | PF_EXITING)))
+		return -EPERM;
+	if (unlikely(!nmi_uaccess_okay()))
+		return -EPERM;
+	if (unlikely(!mm))
+		return -EFAULT;
+	if ((addr & ~PAGE_MASK) || addr > ULONG_MAX - PAGE_SIZE)
+		return -EINVAL;
+
+	if (mmap_write_lock_killable(mm))
+		return -EINTR;
+
+	vma = find_vma(mm, addr);
+	if (!vma || vma->vm_start > addr)
+		goto out;
+	/* Allow to change only vma allocated by bpf_user_page_alloc. */
+	if (vma->vm_private_data != &bpf_user_page_marker)
+		goto out;
+	if (!(vma->vm_flags & VM_WRITE) || !(vma->vm_flags & VM_MAYEXEC))
+		goto out;
+
+	vma_start_write(vma);
+
+	err = get_user_pages(addr, 1, FOLL_WRITE, &page);
+	if (err != 1) {
+		if (!err)
+			err = -EFAULT;
+		goto out;
+	}
+
+	flush_icache_user_page(vma, page, addr, PAGE_SIZE);
+	err = do_mprotect_pkey_locked(addr, PAGE_SIZE, PROT_READ | PROT_EXEC, -1);
+	if (!err) {
+		vma = find_vma(mm, addr);
+		vm_flags_clear(vma, VM_MAYWRITE);
+	}
+	put_page(page);
+out:
+	mmap_write_unlock(mm);
+	return err;
+}
+
 /**
  * bpf_probe_write_user_faultable() - Copy data to userspace with page faults
  * enabled
@@ -5171,6 +5296,8 @@ static const struct btf_kfunc_id_set generic_kfunc_set = {
 };
 
 BTF_KFUNCS_START(user_mem_btf_ids)
+BTF_ID_FLAGS(func, bpf_user_page_alloc, KF_SLEEPABLE | KF_PERFMON)
+BTF_ID_FLAGS(func, bpf_user_page_exec, KF_SLEEPABLE | KF_PERFMON)
 BTF_ID_FLAGS(func, bpf_probe_write_user_faultable, KF_SLEEPABLE | KF_PERFMON)
 BTF_KFUNCS_END(user_mem_btf_ids)
 
Signature · 1 line
2.54.0
#11 lore raw

Re: [RFC bpf-next 4/9] bpf: Add kfuncs to allocate and seal user pages

Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider: - [High] bpf: broken icache flush ordering and sync in bpf_user_page_exec() - [Medium] bpf…

Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:
- [High] bpf: broken icache flush ordering and sync in bpf_user_page_exec()
- [Medium] bpf: READ_IMPLIES_EXEC cleared too late in bpf_user_page_alloc()
- [Medium] bpf: signed long return in bpf_user_page_alloc() is negative for 32-bit user addresses
- [Low] bpf: link failure on !CONFIG_MMU in bpf_user_page_exec()
--

commit bb8df072f0f4bad453d939bfbd2cb461ada4757f
Author: Jiri Olsa <[email protected]>

bpf: Add kfuncs to allocate and seal user pages

This commit adds the sleepable bpf_user_page_alloc() and bpf_user_page_exec() kfuncs for kprobe/uprobe BPF programs to allocate anonymous user pages in the current process and seal individual pages as read/execute.

Quoted 4 lines
kernel/bpf/helpers.c +0 -0
index 4d9e9423826a9..246f4969a144a 100644

[ ... ]

Quoted 48 lines
kernel/bpf/helpers.c +44 -0
@@ -3739,6 +3743,127 @@ __bpf_kfunc int bpf_copy_from_user_mm(void *dst, u32 dst__sz,
 	return ret;
 }
 
+/* vm_private_data token for the unmergeable anonymous mapping below. */
+static char bpf_user_page_marker;
+
+/**
+ * bpf_user_page_alloc() - Allocate lazy anonymous pages in the current mm
+ * @nr_pages: Number of pages to allocate
+ *
+ * Creates a read/write anonymous mapping in the current task's address space.
+ * The mapping is kept non-executable even for tasks with READ_IMPLIES_EXEC,
+ * to avoid creating a writable executable mapping. The physical page is
+ * allocated on its first access. The mapping remains until userspace unmaps
+ * it or the process exits.
+ *
+ * Return: The userspace address on success, or a negative error code.
+ */
+__bpf_kfunc long bpf_user_page_alloc(unsigned long nr_pages)
+{
+	const unsigned long prot = PROT_READ | PROT_WRITE;
+	const unsigned long flags = MAP_PRIVATE | MAP_ANONYMOUS;
+	struct mm_struct *mm = current->mm;
+	unsigned long addr, populate, len;
+	struct vm_area_struct *vma;
+	int err;
+
+	if (unlikely(in_interrupt() || current->flags & (PF_KTHREAD | PF_EXITING)))
+		return -EPERM;
+	if (unlikely(!nmi_uaccess_okay()))
+		return -EPERM;
+	if (unlikely(!mm))
+		return -EFAULT;
+	if (unlikely(!nr_pages || nr_pages > (ULONG_MAX >> PAGE_SHIFT)))
+		return -EINVAL;
+
+	len = nr_pages << PAGE_SHIFT;
+	err = security_mmap_file(NULL, prot, flags);
+	if (err)
+		return err;
+
+	if (mmap_write_lock_killable(mm))
+		return -EINTR;
+
+	addr = do_mmap(NULL, 0, len, prot, flags,
+		       mk_vma_flags(VMA_DONTEXPAND_BIT, VMA_DONTCOPY_BIT, VMA_DONTDUMP_BIT),
+		       0, &populate, NULL);

kernel/bpf/helpers.c __bpf_kfunc int bpf_copy_from_user_mm(void *dst, u32 dst__sz, · line 3799

Quoted 10 lines
kernel/bpf/helpers.c +10 -0
+	if (!IS_ERR_VALUE(addr)) {
+		/* VM_DONTEXPAND prevents merging this mapping into another VMA. */
+		vma = find_vma(mm, addr);
+		vma_start_write(vma);
+		if (current->personality & READ_IMPLIES_EXEC) {
+			vm_flags_clear(vma, VM_EXEC);
+			vma_set_page_prot(vma);
+		}
+		vma->vm_private_data = &bpf_user_page_marker;
+	}

[Severity: Medium]
When current->personality has READ_IMPLIES_EXEC set, is clearing VM_EXEC after do_mmap() too late?

Before bpf_user_page_alloc() clears VM_EXEC on the VMA, both security_mmap_file() and do_mmap() evaluate the request with PROT_EXEC implied:

security/security.c:mmap_prot()

    if (!(current->personality & READ_IMPLIES_EXEC))
        return prot;
    if (!file)
        return prot | PROT_EXEC;

mm/mmap.c:do_mmap()

    if ((prot & PROT_READ) && (current->personality & READ_IMPLIES_EXEC))
        if (!(file && path_noexec(&file->f_path)))
            prot |= PROT_EXEC;
    ...
    mmap_region()
        if (map_deny_write_exec(&vma_flags, &vma_flags))
            return -EACCES;

Because prot becomes PROT_READ | PROT_WRITE | PROT_EXEC during those calls, selinux_mmap_file() checks PROCESS__EXECMEM for an anonymous W+X mapping and map_deny_write_exec() rejects the mapping with -EACCES when MMF_HAS_MDWE is enabled, and perf_event_mmap() emits an executable PERF_RECORD_MMAP event before VM_EXEC is cleared.

kernel/bpf/helpers.c __bpf_kfunc int bpf_copy_from_user_mm(void *dst, u32 dst__sz, · line 3806

Quoted 7 lines
kernel/bpf/helpers.c +7 -0
+
+	mmap_write_unlock(mm);
+
+	if (!IS_ERR_VALUE(addr) && populate)
+		mm_populate(addr, populate);
+	return addr;
+}

[Severity: Medium]
On 32-bit architectures (such as x86-32), can returning addr as a signed long in bpf_user_page_alloc() turn valid userspace addresses into negative values?

do_mmap() returns an unsigned long where valid 32-bit user addresses in [0x80000000, 0xfffff000) have bit 31 set and pass !IS_ERR_VALUE(addr), but returning addr as long makes those addresses negative and indistinguishable from error codes.

In addition, long is 64 bits wide in BPF target programs while long in a 32-bit kernel's vmlinux BTF has ret_size = 4, so emit_kfunc_call() in arch/x86/net/bpf_jit_comp32.c only populates the low 32 bits of R0.

Should bpf_user_page_alloc() return s64 instead, for example returning (long)addr when IS_ERR_VALUE(addr) is true and (s64)addr otherwise?

Quoted 16 lines
kernel/bpf/helpers.c +16 -0
+
+/**
+ * bpf_user_page_exec() - Make a BPF-allocated user page read/execute
+ * @addr: Address of the page to change
+ *
+ * Changes a writable page created by bpf_user_page_alloc() to read/execute.
+ * The page cannot be made writable again after this call.
+ *
+ * Return: 0 on success, or a negative error code.
+ */
+__bpf_kfunc int bpf_user_page_exec(unsigned long addr)
+{
+	struct mm_struct *mm = current->mm;
+	struct vm_area_struct *vma;
+	struct page *page;
+	int err = -EINVAL;

[ ... ]

kernel/bpf/helpers.c __bpf_kfunc int bpf_copy_from_user_mm(void *dst, u32 dst__sz, · line 3855

Quoted 10 lines
kernel/bpf/helpers.c +10 -0
+	vma_start_write(vma);
+
+	err = get_user_pages(addr, 1, FOLL_WRITE, &page);
+	if (err != 1) {
+		if (!err)
+			err = -EFAULT;
+		goto out;
+	}
+
+	flush_icache_user_page(vma, page, addr, PAGE_SIZE);

[Severity: High]
Does calling flush_icache_user_page() in bpf_user_page_exec() before do_mprotect_pkey_locked() leave stale instructions in the instruction cache?

At the point where flush_icache_user_page() is called, the VMA still has VM_READ | VM_WRITE without VM_EXEC. Several architectures gate flush_icache_user_page() on VM_EXEC and turn this call into a no-op:

arch/riscv/include/asm/cacheflush.h:

    #define flush_icache_user_page(vma, pg, addr, len) \
    do {                                               \
        if (vma->vm_flags & VM_EXEC)                   \
            flush_icache_mm(vma->vm_mm, 0);            \
    } while (0)

arch/openrisc/include/asm/cacheflush.h:

    #define flush_icache_user_page(vma, page, addr, len) \
    do {                                                 \
        if (vma->vm_flags & VM_EXEC)                     \
            sync_icache_dcache(page);                    \
    } while (0)

arch/alpha/kernel/smp.c:flush_icache_user_page()

    if ((vma->vm_flags & VM_EXEC) == 0)
        return;

In addition:

1. Because flush_icache_user_page() runs while the PTE is still writable and before do_mprotect_pkey_locked() write-protects the PTE and flushes remote TLBs, another thread in current->mm can write to addr after the icache flush and before write access is removed.

2. On arm64 and riscv, mprotect's set_ptes() path only synchronizes caches when PG_dcache_clean is clear on the folio and then sets PG_dcache_clean (for example in arch/arm64/mm/flush.c:__sync_icache_dcache()). If a multi-page allocation from bpf_user_page_alloc() is backed by a multi-page anonymous folio (mTHP), sealing a second modified subpage of the same folio will see PG_dcache_clean already set and skip cache maintenance unless flush_dcache_page(page) is called to clear PG_dcache_clean.

3. On LoongArch (arch/loongarch/include/asm/cacheflush.h), both flush_icache_user_page() and flush_dcache_page() are no-ops and set_pte() does not emit an instruction barrier.

kernel/bpf/helpers.c __bpf_kfunc int bpf_copy_from_user_mm(void *dst, u32 dst__sz, · line 3856

Quoted 1 line
kernel/bpf/helpers.c +1 -0
+	err = do_mprotect_pkey_locked(addr, PAGE_SIZE, PROT_READ | PROT_EXEC, -1);

[Severity: Low]
Will calling do_mprotect_pkey_locked() unconditionally in bpf_user_page_exec() fail to link when CONFIG_BPF_SYSCALL=y and CONFIG_MMU=n?

kernel/bpf/helpers.c is built whenever CONFIG_BPF_SYSCALL=y, which does not depend on CONFIG_MMU, whereas do_mprotect_pkey_locked() is only defined in mm/mprotect.c when CONFIG_MMU=y:

mm/Makefile:

    mmu-$(CONFIG_MMU) := highmem.o memory.o mincore.o \
                         mlock.o mmap.o mmu_gather.o mprotect.o mremap.o \

include/linux/mm.h declares do_mprotect_pkey_locked() without a !CONFIG_MMU stub, and mm/nommu.c:do_mmap() allocates non-lazy physical memory that cannot be transitioned to read/execute via page tables.

Quoted 9 lines
kernel/bpf/helpers.c +9 -0
+	if (!err) {
+		vma = find_vma(mm, addr);
+		vm_flags_clear(vma, VM_MAYWRITE);
+	}
+	put_page(page);
+out:
+	mmap_write_unlock(mm);
+	return err;
+}

[ ... ]

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
#12 lore raw

Re: [RFC bpf-next 4/9] bpf: Add kfuncs to allocate and seal user pages

In reply to Jiri Olsa
TL;DR: bpf_user_page_exec() may skip the icache flush on arm64/riscv, since flush_icache_user_page() runs before VM_EXEC is set, so with THP a task could execute stale instructions.
Quoted 1 line

bpf: Add kfuncs to allocate and seal user pages

TL;DR: bpf_user_page_exec() may skip the icache flush on arm64/riscv, since flush_icache_user_page() runs before VM_EXEC is set, so with THP a task could execute stale instructions.

Quoted 4 lines
kernel/bpf/helpers.c +0 -0
index 4d9e9423826a..246f4969a144 100644

[ ... ]

Quoted 1 line
kernel/bpf/helpers.c +0 -0
@@ -3739,6 +3743,127 @@ __bpf_kfunc int bpf_copy_from_user_mm(void *dst, u32 dst__sz,

[ ... ]

Quoted 2 lines
kernel/bpf/helpers.c +2 -0
+__bpf_kfunc int bpf_user_page_exec(unsigned long addr)
+{

[ ... ]

Quoted 1 line
kernel/bpf/helpers.c +1 -0
+	err = get_user_pages(addr, 1, FOLL_WRITE, &page);

[ ... ]

kernel/bpf/helpers.c __bpf_kfunc int bpf_copy_from_user_mm(void *dst, u32 dst__sz, · line 3856

Quoted 2 lines
kernel/bpf/helpers.c +2 -0
+	flush_icache_user_page(vma, page, addr, PAGE_SIZE);
+	err = do_mprotect_pkey_locked(addr, PAGE_SIZE, PROT_READ | PROT_EXEC, -1);

Does flush_icache_user_page() do anything here on the architectures that need it?

At this point the VMA is still the read/write mapping created by bpf_user_page_alloc(), so VM_EXEC is not set yet. On riscv the macro is:

    #define flush_icache_user_page(vma, pg, addr, len)  \
    do {                                                \
            if (vma->vm_flags & VM_EXEC)                \
                    flush_icache_mm(vma->vm_mm, 0);     \
    } while (0)

so it is skipped, and VM_EXEC only gets set by the do_mprotect_pkey_locked() call on the next line. arm64 does not define flush_icache_user_page(), so it falls back to the empty stub in include/asm-generic/cacheflush.h.

That leaves only the set_ptes() hook that runs when change_pte_range() installs the read/execute PTE. On arm64 it is __sync_cache_and_tags()->__sync_icache_dcache(), and on riscv it is __set_pte_at()->flush_icache_pte(). Both only do cache maintenance when PG_dcache_clean is clear on the folio, and then set it for the whole folio:

    if (!test_bit(PG_dcache_clean, &folio->flags.f)) {
            sync_icache_aliases(folio_address(folio),
                                folio_address(folio) + folio_size(folio));
            set_bit(PG_dcache_clean, &folio->flags.f);
    }

The BPF program writes the instructions through the user address with bpf_probe_write_user_faultable(), which uses copy_to_user() and never calls flush_dcache_folio(), so PG_dcache_clean is not cleared again after the first exec mapping of the folio.

With THP enabled (arm64 or riscv, 4K pages), can this sequence happen?

bpf_user_page_alloc(512) returns a PMD-aligned address
the program writes page 0, and the fault maps a 2MB anonymous THP bpf_user_page_exec(addr) splits the VMA and the PMD, the read/execute
set_ptes() for page 0 syncs the whole 2MB folio and sets PG_dcache_clean
the program writes code into page 1, which is still read/write bpf_user_page_exec(addr + PAGE_SIZE) calls flush_icache_user_page() again
as a no-op, and the read/execute set_ptes() sees PG_dcache_clean set and skips maintenance

The task would then execute page 1 without a D-cache clean to the point of unification and I-cache invalidate (arm64 without CTR_EL0.IDC/DIC), or without fence.i (riscv), and could fetch stale instructions. mTHP sizes give the same result for smaller allocations.

The uprobe XOL code, which also writes instructions for user space, does explicit maintenance through the kernel alias: arm64 arch_uprobe_copy_ixol() calls sync_icache_aliases(), and riscv, loongarch and mips call flush_icache_range() on the kmap address.

Should bpf_user_page_exec() do the same, for example with kmap_local_page(page) followed by flush_icache_range()? Alternatively, would calling flush_dcache_page(page) before the mprotect, so the arch set_ptes() hook redoes the sync, be enough? Either way the flush would no longer depend on VM_EXEC being set at the time of the call.

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/39668036298

#13 lore raw

Re: [RFC bpf-next 4/9] bpf: Add kfuncs to allocate and seal user pages

In reply to Jiri Olsa
kernel test robot noticed the following build errors:

Hi Jiri,

[This is a private test report for your RFC patch.] kernel test robot noticed the following build errors:

[auto build test ERROR on bpf-next/master]

url:    https://github.com/intel-lab-lkp/linux/commits/Jiri-Olsa/bpf-Add-__srcptr-kfunc-argument-suffix/20261021-215750
base:   https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git master
patch link:    https://lore.kernel.org/r/20261021195759.119421-5-jolsa%40kernel.org
patch subject: [RFC bpf-next 4/9] bpf: Add kfuncs to allocate and seal user pages
config: nios2-allmodconfig (https://download.01.org/0day-ci/archive/20261026/[email protected]/config)
compiler: nios2-linux-gcc (GCC) 11.5.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20261026/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All errors (new ones prefixed by >>):

kernel/bpf/helpers.c: In function 'bpf_user_page_exec':

>> kernel/bpf/helpers.c:3855:9: error: implicit declaration of function 'flush_icache_user_page'; did you mean 'flush_icache_page'? [-Werror=implicit-function-declaration]
    3855 |         flush_icache_user_page(vma, page, addr, PAGE_SIZE);
         |         ^~~~~~~~~~~~~~~~~~~~~~
         |         flush_icache_page

cc1: some warnings being treated as errors

vim +3855 kernel/bpf/helpers.c

3807
3808 /**
3809 * bpf_user_page_exec() - Make a BPF-allocated user page read/execute
3810 * @addr: Address of the page to change
3811 *
3812 * Changes a writable page created by bpf_user_page_alloc() to read/execute.
3813 * The page cannot be made writable again after this call.
3814 *
3815 * Return: 0 on success, or a negative error code.
3816 */
3817 __bpf_kfunc int bpf_user_page_exec(unsigned long addr)
3818 {
3819 struct mm_struct *mm = current->mm;
3820 struct vm_area_struct *vma;
3821 struct page *page;
3822 int err = -EINVAL;
3823
3824 if (unlikely(in_interrupt() ||
3825 current->flags & (PF_KTHREAD | PF_EXITING)))
3826 return -EPERM;
3827 if (unlikely(!nmi_uaccess_okay()))
3828 return -EPERM;
3829 if (unlikely(!mm))
3830 return -EFAULT;
3831 if ((addr & ~PAGE_MASK) || addr > ULONG_MAX - PAGE_SIZE)
3832 return -EINVAL;
3833
3834 if (mmap_write_lock_killable(mm))
3835 return -EINTR;
3836
3837 vma = find_vma(mm, addr);
3838 if (!vma || vma->vm_start > addr)
3839 goto out;
3840 /* Allow to change only vma allocated by bpf_user_page_alloc. */
3841 if (vma->vm_private_data != &bpf_user_page_marker)
3842 goto out;
3843 if (!(vma->vm_flags & VM_WRITE) || !(vma->vm_flags & VM_MAYEXEC))
3844 goto out;
3845
3846 vma_start_write(vma);
3847
3848 err = get_user_pages(addr, 1, FOLL_WRITE, &page);
3849 if (err != 1) {
3850 if (!err)
3851 err = -EFAULT;
3852 goto out;
3853 }
3854

> 3855		flush_icache_user_page(vma, page, addr, PAGE_SIZE);

3856 err = do_mprotect_pkey_locked(addr, PAGE_SIZE, PROT_READ | PROT_EXEC, -1);
3857 if (!err) {
3858 vma = find_vma(mm, addr);
3859 vm_flags_clear(vma, VM_MAYWRITE);
3860 }
3861 put_page(page);
3862 out:
3863 mmap_write_unlock(mm);
3864 return err;
3865 }
3866

--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki

#14 lore raw

Re: [RFC bpf-next 4/9] bpf: Add kfuncs to allocate and seal user pages

In reply to Jiri Olsa
kernel test robot noticed the following build errors:

Hi Jiri,

[This is a private test report for your RFC patch.] kernel test robot noticed the following build errors:

[auto build test ERROR on bpf-next/master]

url:    https://github.com/intel-lab-lkp/linux/commits/Jiri-Olsa/bpf-Add-__srcptr-kfunc-argument-suffix/20261021-215750
base:   https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git master
patch link:    https://lore.kernel.org/r/20261021195759.119421-5-jolsa%40kernel.org
patch subject: [RFC bpf-next 4/9] bpf: Add kfuncs to allocate and seal user pages
config: sparc64-allmodconfig (https://download.01.org/0day-ci/archive/20261026/[email protected]/config)
compiler: clang version 20.1.8 (https://github.com/llvm/llvm-project 87f0227cb60147a26a1eeb4fb06e3b505e9c7261)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20261026/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All errors (new ones prefixed by >>):

>> kernel/bpf/helpers.c:3855:2: error: call to undeclared function 'flush_icache_user_page'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
    3855 |         flush_icache_user_page(vma, page, addr, PAGE_SIZE);
         |         ^

1 error generated.

vim +/flush_icache_user_page +3855 kernel/bpf/helpers.c

3807
3808 /**
3809 * bpf_user_page_exec() - Make a BPF-allocated user page read/execute
3810 * @addr: Address of the page to change
3811 *
3812 * Changes a writable page created by bpf_user_page_alloc() to read/execute.
3813 * The page cannot be made writable again after this call.
3814 *
3815 * Return: 0 on success, or a negative error code.
3816 */
3817 __bpf_kfunc int bpf_user_page_exec(unsigned long addr)
3818 {
3819 struct mm_struct *mm = current->mm;
3820 struct vm_area_struct *vma;
3821 struct page *page;
3822 int err = -EINVAL;
3823
3824 if (unlikely(in_interrupt() ||
3825 current->flags & (PF_KTHREAD | PF_EXITING)))
3826 return -EPERM;
3827 if (unlikely(!nmi_uaccess_okay()))
3828 return -EPERM;
3829 if (unlikely(!mm))
3830 return -EFAULT;
3831 if ((addr & ~PAGE_MASK) || addr > ULONG_MAX - PAGE_SIZE)
3832 return -EINVAL;
3833
3834 if (mmap_write_lock_killable(mm))
3835 return -EINTR;
3836
3837 vma = find_vma(mm, addr);
3838 if (!vma || vma->vm_start > addr)
3839 goto out;
3840 /* Allow to change only vma allocated by bpf_user_page_alloc. */
3841 if (vma->vm_private_data != &bpf_user_page_marker)
3842 goto out;
3843 if (!(vma->vm_flags & VM_WRITE) || !(vma->vm_flags & VM_MAYEXEC))
3844 goto out;
3845
3846 vma_start_write(vma);
3847
3848 err = get_user_pages(addr, 1, FOLL_WRITE, &page);
3849 if (err != 1) {
3850 if (!err)
3851 err = -EFAULT;
3852 goto out;
3853 }
3854

> 3855		flush_icache_user_page(vma, page, addr, PAGE_SIZE);

3856 err = do_mprotect_pkey_locked(addr, PAGE_SIZE, PROT_READ | PROT_EXEC, -1);
3857 if (!err) {
3858 vma = find_vma(mm, addr);
3859 vm_flags_clear(vma, VM_MAYWRITE);
3860 }
3861 put_page(page);
3862 out:
3863 mmap_write_unlock(mm);
3864 return err;
3865 }
3866

--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki

author reviewer
#15 lore raw

RFC 5/9 selftests/bpf: Add __srcptr kfunc arguments test

In reply to Jiri Olsa

Add positive kfunc tests for read-only global and writable stack sources annotated with __srcptr. Also verify that a read-only source is rejected when passed to a writable mem_len argument.

Assisted-by Codex:GPT-6
---
 tools/testing/selftests/bpf/prog_tests/kfunc_call.c  |  2 ++
 tools/testing/selftests/bpf/progs/kfunc_call_fail.c  | 10 ++++++++++
 tools/testing/selftests/bpf/progs/kfunc_call_test.c  | 12 ++++++++++++
 tools/testing/selftests/bpf/test_kmods/bpf_testmod.c |  5 +++++
 .../selftests/bpf/test_kmods/bpf_testmod_kfunc.h     |  1 +
 5 files changed, 30 insertions(+)
tools/testing/selftests/bpf/prog_tests/kfunc_call.c +2 -0
index 0063e60d6f2f..9b832c5961f5 100644
@@ -63,6 +63,7 @@ static struct kfunc_test_params kfunc_tests[] = {
 	 */
 	SYSCALL_NULL_CTX_FAIL(kfunc_syscall_test_fail, -EINVAL, "processed 4 insns"),
 	SYSCALL_NULL_CTX_FAIL(kfunc_syscall_test_null_fail, -EINVAL, "processed 4 insns"),
+	TC_FAIL(kfunc_call_test_mem_len_fail_readonly, 0, "write into map forbidden"),
 	TC_FAIL(kfunc_call_test_get_mem_fail_rdonly, 0, "R0 cannot write into rdonly_mem"),
 	TC_FAIL(kfunc_call_test_get_mem_fail_use_after_free, 0, "invalid mem access 'scalar'"),
 	TC_FAIL(kfunc_call_test_get_mem_fail_oob, 0, "min value is outside of the allowed memory range"),
@@ -82,6 +83,7 @@ static struct kfunc_test_params kfunc_tests[] = {
 	TC_TEST(kfunc_call_test5_asm, 0),
 	TC_TEST(kfunc_call_test_ref_btf_id, 0),
 	TC_TEST(kfunc_call_test_get_mem, 42),
+	TC_TEST(kfunc_call_test_srcptr, 0),
 	SYSCALL_TEST(kfunc_syscall_test, 0),
 	SYSCALL_NULL_CTX_TEST(kfunc_syscall_test_null, 0),
 	TC_TEST(kfunc_call_test_static_unused_arg, 0),
tools/testing/selftests/bpf/progs/kfunc_call_fail.c +10 -0
index 7e93f7fb1329..1ab85faaf8db 100644
@@ -4,6 +4,8 @@
 #include <bpf/bpf_helpers.h>
 #include "../test_kmods/bpf_testmod_kfunc.h"
 
+static const char mem_len_rodata[] = "read-only data";
+
 static struct bpf_spin_lock kfunc_call_lock SEC(".data.A");
 
 SEC("?tc")
@@ -47,6 +49,14 @@ int kfunc_syscall_test_null_fail(struct syscall_test_args *args)
 	return 0;
 }
 
+SEC("?tc")
+int kfunc_call_test_mem_len_fail_readonly(struct __sk_buff *skb)
+{
+	bpf_kfunc_call_test_mem_len_pass1((void *)mem_len_rodata,
+					  sizeof(mem_len_rodata));
+	return 0;
+}
+
 SEC("?tc")
 int kfunc_call_test_get_mem_fail_rdonly(struct __sk_buff *skb)
 {
tools/testing/selftests/bpf/progs/kfunc_call_test.c +12 -0
index 8e6560c31e78..b944e2e186b6 100644
@@ -5,6 +5,8 @@
 #include "bpf_misc.h"
 #include "../test_kmods/bpf_testmod_kfunc.h"
 
+static const char srcptr_rodata[] = "read-only source";
+
 static struct bpf_spin_lock kfunc_call_lock SEC(".data.A");
 
 SEC("tc")
@@ -208,6 +210,16 @@ int kfunc_call_test_pass(struct __sk_buff *skb)
 	return 0;
 }
 
+SEC("tc")
+int kfunc_call_test_srcptr(struct __sk_buff *skb)
+{
+	char stack_data[] = "read-write source";
+
+	bpf_kfunc_call_test_srcptr(srcptr_rodata, sizeof(srcptr_rodata));
+	bpf_kfunc_call_test_srcptr(stack_data, sizeof(stack_data));
+	return 0;
+}
+
 struct syscall_test_args {
 	__u8 data[16];
 	size_t size;
tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +5 -0
index 6380f52f4dd2..5d72f94a8912 100644
@@ -1024,6 +1024,7 @@ BTF_ID_FLAGS(func, bpf_kfunc_arena_args5_test)
 BTF_ID_FLAGS(func, bpf_kfunc_arena_stack_arg_test)
 BTF_ID_FLAGS(func, bpf_kfunc_arena_mixed_test)
 BTF_ID_FLAGS(func, bpf_kfunc_call_test_mem_len_pass1)
+BTF_ID_FLAGS(func, bpf_kfunc_call_test_srcptr)
 BTF_ID_FLAGS(func, bpf_kfunc_dynptr_test)
 BTF_ID_FLAGS(func, bpf_kfunc_nested_acquire_nonzero_offset_test, KF_ACQUIRE)
 BTF_ID_FLAGS(func, bpf_kfunc_nested_acquire_zero_offset_test, KF_ACQUIRE)
@@ -1527,6 +1528,10 @@ __bpf_kfunc void bpf_kfunc_call_test_mem_len_pass1(void *mem, int mem__sz)
 {
 }
 
+__bpf_kfunc void bpf_kfunc_call_test_srcptr(const void *src__srcptr, int src__sz)
+{
+}
+
 __bpf_kfunc void bpf_kfunc_call_test_mem_len_fail1(void *mem, int len)
 {
 }
tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h +1 -0
index 3b810f1e438f..919c4c450ef1 100644
@@ -198,6 +198,7 @@ void bpf_kfunc_call_test_release(struct prog_test_ref_kfunc *p) __ksym;
 void bpf_kfunc_call_test_ref(struct prog_test_ref_kfunc *p) __ksym;
 
 void bpf_kfunc_call_test_mem_len_pass1(void *mem, int len) __ksym;
+void bpf_kfunc_call_test_srcptr(const void *src__srcptr, int src__sz) __ksym;
 __u64 bpf_kfunc_arena_arg_test(__u64 *val__arena) __ksym;
 __u64 bpf_kfunc_arena_cap_test(__u64 *val__arena) __ksym;
 __u64 bpf_kfunc_arena_cap_nullable_test(__u64 *val__arena__nullable) __ksym;
Signature · 1 line
2.54.0
author reviewer
#16 lore raw

RFC 6/9 selftests/bpf: Add user page allocation test

In reply to Jiri Olsa

Check that consecutive allocations remain separate VMAs and that user page can be allocated again after unmapping.

Assisted-by Codex:GPT-6
---
 .../selftests/bpf/prog_tests/user_page.c      | 152 ++++++++++++++++++
 tools/testing/selftests/bpf/progs/user_page.c |  31 ++++
 2 files changed, 183 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/user_page.c
 create mode 100644 tools/testing/selftests/bpf/progs/user_page.c
tools/testing/selftests/bpf/prog_tests/user_page.c new file +152 -0
new file mode 100644
index 000000000000..ff51e7c598a9
@@ -0,0 +1,152 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <test_progs.h>
+#include <sys/mman.h>
+#include <sys/personality.h>
+#include <unistd.h>
+
+#include "user_page.skel.h"
+
+enum {
+	USER_PAGE_ALLOC = 1,
+};
+
+static noinline void user_page_trigger(void)
+{
+	asm volatile ("");
+}
+
+static bool check_user_page_vma(void *addr, long size, const char *perms)
+{
+	unsigned long start, end;
+	bool found = false;
+	char vma_perms[5];
+	char line[512];
+	FILE *maps;
+
+	maps = fopen("/proc/self/maps", "r");
+	if (!ASSERT_OK_PTR(maps, "open_maps"))
+		return false;
+
+	while (fgets(line, sizeof(line), maps)) {
+		if (sscanf(line, "%lx-%lx %4s", &start, &end, vma_perms) != 3 ||
+		    (unsigned long)addr < start || (unsigned long)addr >= end)
+			continue;
+		found = ASSERT_EQ(start, (unsigned long)addr, "vma_start") &&
+			ASSERT_EQ(end - start, size, "vma_size") &&
+			(!perms || ASSERT_STREQ(vma_perms, perms, "vma_perms"));
+		break;
+	}
+	fclose(maps);
+	return ASSERT_TRUE(found, "user_page_vma");
+}
+
+static bool alloc_user_pages(struct user_page *skel, unsigned long nr_pages,
+			     long *addr)
+{
+	skel->bss->nr_pages = nr_pages;
+	skel->bss->action = USER_PAGE_ALLOC;
+	user_page_trigger();
+	if (!ASSERT_GE(skel->bss->user_addr, 0, "user_addr"))
+		return false;
+
+	*addr = skel->bss->user_addr;
+	return true;
+}
+
+static void test_allocations(struct user_page *skel)
+{
+	void *pages[2] = { MAP_FAILED, MAP_FAILED };
+	long page_size = sysconf(_SC_PAGESIZE);
+	const unsigned long nr_pages = 3;
+	long addr;
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(pages); i++) {
+		if (!alloc_user_pages(skel, nr_pages, &addr))
+			goto out;
+		pages[i] = (void *)addr;
+	}
+
+	/* Back-to-back allocations must remain individual VMAs. */
+	for (i = 0; i < ARRAY_SIZE(pages); i++) {
+		if (!check_user_page_vma(pages[i], page_size * nr_pages, NULL))
+			break;
+	}
+
+out:
+	for (i = 0; i < ARRAY_SIZE(pages); i++)
+		if (pages[i] != MAP_FAILED)
+			munmap(pages[i], page_size * nr_pages);
+}
+
+static void test_reallocate_after_unmap(struct user_page *skel)
+{
+	long page_size = sysconf(_SC_PAGESIZE);
+	long addr = -1;
+
+	if (!alloc_user_pages(skel, 1, &addr))
+		goto out;
+
+	if (!ASSERT_OK(munmap((void *)addr, page_size), "munmap"))
+		goto out;
+
+	addr = -1;
+	alloc_user_pages(skel, 1, &addr);
+
+out:
+	if (addr >= 0)
+		munmap((void *)addr, page_size);
+}
+
+static void test_read_implies_exec(struct user_page *skel)
+{
+	long page_size = sysconf(_SC_PAGESIZE);
+	long addr = -1;
+	int old_personality, set_personality;
+
+	old_personality = personality(0xffffffffUL);
+	if (!ASSERT_NEQ(old_personality, -1, "get_personality"))
+		return;
+
+	set_personality = old_personality | READ_IMPLIES_EXEC;
+	if (!ASSERT_NEQ(personality(set_personality), -1, "set_personality"))
+		return;
+
+	skel->bss->action = USER_PAGE_ALLOC;
+	skel->bss->nr_pages = 1;
+	user_page_trigger();
+	if (!ASSERT_GE(skel->bss->user_addr, 0, "read_implies_exec_alloc_ret"))
+		goto out;
+	addr = skel->bss->user_addr;
+	check_user_page_vma((void *)addr, page_size, "rw-p");
+
+out:
+	if (addr >= 0)
+		munmap((void *)addr, page_size);
+	ASSERT_EQ(personality(old_personality), set_personality,
+		  "restore_personality");
+}
+
+void test_user_page(void)
+{
+	struct user_page *skel;
+
+	skel = user_page__open_and_load();
+	if (!ASSERT_OK_PTR(skel, "open_and_load"))
+		return;
+
+	skel->bss->monitored_pid = getpid();
+	if (!ASSERT_OK(user_page__attach(skel), "attach"))
+		goto out;
+
+	if (test__start_subtest("allocations"))
+		test_allocations(skel);
+	if (test__start_subtest("reallocate_after_unmap"))
+		test_reallocate_after_unmap(skel);
+	if (test__start_subtest("read_implies_exec"))
+		test_read_implies_exec(skel);
+
+out:
+	user_page__destroy(skel);
+}
tools/testing/selftests/bpf/progs/user_page.c new file +31 -0
new file mode 100644
index 000000000000..6dd9417463df
@@ -0,0 +1,31 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+
+#include "bpf_misc.h"
+
+char _license[] SEC("license") = "GPL";
+
+#define USER_PAGE_ALLOC 1
+
+int monitored_pid;
+int action;
+long user_addr;
+unsigned long nr_pages;
+
+SEC("uprobe.s//proc/self/exe:user_page_trigger")
+int user_page_test(struct pt_regs *ctx)
+{
+	if ((bpf_get_current_pid_tgid() >> 32) != monitored_pid)
+		return 0;
+
+	switch (action) {
+	case USER_PAGE_ALLOC:
+		user_addr = bpf_user_page_alloc(nr_pages);
+		break;
+	}
+
+	return 0;
+}
Signature · 1 line
2.54.0
author reviewer
#17 lore raw

RFC 7/9 selftests/bpf: Add uprobe user page exec test

In reply to Jiri Olsa

Test executable permissions for whole and partial user-page allocations and execution of code written into a user page.

Assisted-by Codex:GPT-6
---
 .../selftests/bpf/prog_tests/user_page.c      | 62 +++++++++++++++++++
 tools/testing/selftests/bpf/progs/user_page.c | 18 ++++++
 2 files changed, 80 insertions(+)
tools/testing/selftests/bpf/prog_tests/user_page.c +62 -0
index ff51e7c598a9..1b5dff2c8ff6 100644
@@ -9,6 +9,8 @@
 
 enum {
 	USER_PAGE_ALLOC = 1,
+	USER_PAGE_EXEC = 2,
+	USER_PAGE_EXEC_ONLY = 3,
 };
 
 static noinline void user_page_trigger(void)
@@ -128,6 +130,62 @@ static void test_read_implies_exec(struct user_page *skel)
 		  "restore_personality");
 }
 
+static void test_execute_user_page(struct user_page *skel)
+{
+	long page_size = sysconf(_SC_PAGESIZE);
+	long addr = -1;
+
+#ifndef __x86_64__
+	test__skip();
+	return;
+#endif
+
+	if (!alloc_user_pages(skel, 1, &addr))
+		return;
+
+	skel->bss->exec_addr = addr;
+	skel->bss->action = USER_PAGE_EXEC;
+	user_page_trigger();
+	if (!ASSERT_EQ(skel->bss->write_ret, 0, "exec_write_ret") ||
+	    !ASSERT_EQ(skel->bss->exec_ret, 0, "exec_ret"))
+		goto out;
+
+	ASSERT_EQ(((int (*)(void))addr)(), 42, "execute_user_page");
+
+out:
+	if (addr >= 0)
+		munmap((void *)addr, page_size);
+}
+
+static void test_partial_exec(struct user_page *skel)
+{
+	long page_size = sysconf(_SC_PAGESIZE);
+	long addr = -1;
+
+	if (!alloc_user_pages(skel, 3, &addr))
+		return;
+
+	skel->bss->exec_addr = addr + page_size;
+	skel->bss->action = USER_PAGE_EXEC_ONLY;
+	user_page_trigger();
+	if (!ASSERT_OK(skel->bss->exec_ret, "middle_page_exec_ret") ||
+	    !check_user_page_vma((void *)(addr + 0 * page_size), page_size, "rw-p") ||
+	    !check_user_page_vma((void *)(addr + 1 * page_size), page_size, "r-xp") ||
+	    !check_user_page_vma((void *)(addr + 2 * page_size), page_size, "rw-p"))
+		goto out;
+
+	skel->bss->exec_addr = addr + 2 * page_size;
+	user_page_trigger();
+	if (!ASSERT_OK(skel->bss->exec_ret, "last_page_exec_ret"))
+		goto out;
+
+	check_user_page_vma((void *)(addr + 2 * page_size), page_size, "r-xp");
+
+out:
+	if (addr >= 0)
+		munmap((void *)addr, page_size * 3);
+}
+
 void test_user_page(void)
 {
 	struct user_page *skel;
@@ -146,6 +204,10 @@ void test_user_page(void)
 		test_reallocate_after_unmap(skel);
 	if (test__start_subtest("read_implies_exec"))
 		test_read_implies_exec(skel);
+	if (test__start_subtest("execute_user_page"))
+		test_execute_user_page(skel);
+	if (test__start_subtest("partial_exec"))
+		test_partial_exec(skel);
 
 out:
 	user_page__destroy(skel);
tools/testing/selftests/bpf/progs/user_page.c +18 -0
index 6dd9417463df..38f6fcb89bc6 100644
@@ -9,11 +9,20 @@
 char _license[] SEC("license") = "GPL";
 
 #define USER_PAGE_ALLOC 1
+#define USER_PAGE_EXEC 2
+#define USER_PAGE_EXEC_ONLY 3
 
 int monitored_pid;
 int action;
 long user_addr;
+long exec_addr;
 unsigned long nr_pages;
+int write_ret;
+int exec_ret;
+static const unsigned char insns[] = {
+	0xf3, 0x0f, 0x1e, 0xfa, /* endbr64 */
+	0xb8, 42, 0, 0, 0, 0xc3, /* mov $42, %eax; ret */
+};
 
 SEC("uprobe.s//proc/self/exe:user_page_trigger")
 int user_page_test(struct pt_regs *ctx)
@@ -25,6 +34,15 @@ int user_page_test(struct pt_regs *ctx)
 	case USER_PAGE_ALLOC:
 		user_addr = bpf_user_page_alloc(nr_pages);
 		break;
+	case USER_PAGE_EXEC:
+		write_ret = bpf_probe_write_user_faultable((void *)exec_addr,
+							   insns, sizeof(insns));
+		exec_ret = write_ret ? write_ret :
+			bpf_user_page_exec(exec_addr);
+		break;
+	case USER_PAGE_EXEC_ONLY:
+		exec_ret = bpf_user_page_exec(exec_addr);
+		break;
 	}
 
 	return 0;
Signature · 1 line
2.54.0
author reviewer
#18 lore raw

RFC 8/9 selftests/bpf: Add faultable user page writes test

In reply to Jiri Olsa

Exercise fault-enabled writes into an allocated user page and verify the written data.

Assisted-by Codex:GPT-6
---
 .../selftests/bpf/prog_tests/user_page.c      | 25 +++++++++++++++++++
 tools/testing/selftests/bpf/progs/user_page.c |  7 ++++++
 2 files changed, 32 insertions(+)
tools/testing/selftests/bpf/prog_tests/user_page.c +25 -0
index 1b5dff2c8ff6..74ce52d726ec 100644
@@ -11,6 +11,7 @@ enum {
 	USER_PAGE_ALLOC = 1,
 	USER_PAGE_EXEC = 2,
 	USER_PAGE_EXEC_ONLY = 3,
+	USER_PAGE_WRITE = 4,
 };
 
 static noinline void user_page_trigger(void)
@@ -186,6 +187,28 @@ static void test_partial_exec(struct user_page *skel)
 		munmap((void *)addr, page_size * 3);
 }
 
+static void test_faultable_write(struct user_page *skel)
+{
+	static const char expected[] = "fault-enabled BPF write";
+	long page_size = sysconf(_SC_PAGESIZE);
+	long addr = -1;
+
+	if (!alloc_user_pages(skel, 1, &addr))
+		return;
+
+	skel->bss->user_addr = addr;
+	skel->bss->action = USER_PAGE_WRITE;
+	user_page_trigger();
+	if (!ASSERT_EQ(skel->bss->write_ret, 0, "write_ret"))
+		goto out;
+
+	ASSERT_EQ(memcmp((void *)addr, expected, sizeof(expected)), 0, "written_data");
+
+out:
+	if (addr >= 0)
+		munmap((void *)addr, page_size);
+}
+
 void test_user_page(void)
 {
 	struct user_page *skel;
@@ -208,6 +231,8 @@ void test_user_page(void)
 		test_execute_user_page(skel);
 	if (test__start_subtest("partial_exec"))
 		test_partial_exec(skel);
+	if (test__start_subtest("faultable_write"))
+		test_faultable_write(skel);
 
 out:
 	user_page__destroy(skel);
tools/testing/selftests/bpf/progs/user_page.c +7 -0
index 38f6fcb89bc6..a855b0b46350 100644
@@ -11,6 +11,7 @@ char _license[] SEC("license") = "GPL";
 #define USER_PAGE_ALLOC 1
 #define USER_PAGE_EXEC 2
 #define USER_PAGE_EXEC_ONLY 3
+#define USER_PAGE_WRITE 4
 
 int monitored_pid;
 int action;
@@ -19,6 +20,8 @@ long exec_addr;
 unsigned long nr_pages;
 int write_ret;
 int exec_ret;
+
+static const char data[] = "fault-enabled BPF write";
 static const unsigned char insns[] = {
 	0xf3, 0x0f, 0x1e, 0xfa, /* endbr64 */
 	0xb8, 42, 0, 0, 0, 0xc3, /* mov $42, %eax; ret */
@@ -43,6 +46,10 @@ int user_page_test(struct pt_regs *ctx)
 	case USER_PAGE_EXEC_ONLY:
 		exec_ret = bpf_user_page_exec(exec_addr);
 		break;
+	case USER_PAGE_WRITE:
+		write_ret = bpf_probe_write_user_faultable((void *)user_addr, data,
+							   sizeof(data));
+		break;
 	}
 
 	return 0;
Signature · 1 line
2.54.0
author reviewer
#19 lore raw

RFC 9/9 selftests/bpf: Add user page failures test

In reply to Jiri Olsa

Cover expected failures for invalid allocation sizes, misaligned or unrelated exec requests, replacement of sealed mappings, repeated execution, bad writes and attempts to restore write permission.

Assisted-by Codex:GPT-6
---
 .../selftests/bpf/prog_tests/user_page.c      | 141 ++++++++++++++++++
 tools/testing/selftests/bpf/progs/user_page.c |   5 +
 2 files changed, 146 insertions(+)
tools/testing/selftests/bpf/prog_tests/user_page.c +141 -0
index 74ce52d726ec..107349aace94 100644
@@ -12,6 +12,7 @@ enum {
 	USER_PAGE_EXEC = 2,
 	USER_PAGE_EXEC_ONLY = 3,
 	USER_PAGE_WRITE = 4,
+	USER_PAGE_BAD_WRITE = 5,
 };
 
 static noinline void user_page_trigger(void)
@@ -209,6 +210,144 @@ static void test_faultable_write(struct user_page *skel)
 		munmap((void *)addr, page_size);
 }
 
+static void test_invalid_allocation_size(struct user_page *skel)
+{
+	skel->bss->action = USER_PAGE_ALLOC;
+	skel->bss->nr_pages = 0;
+	user_page_trigger();
+	if (!ASSERT_EQ(skel->bss->user_addr, -EINVAL, "zero_pages_ret"))
+		return;
+
+	skel->bss->nr_pages = ~0UL;
+	user_page_trigger();
+	ASSERT_EQ(skel->bss->user_addr, -EINVAL, "overflow_pages_ret");
+}
+
+static void test_misaligned_exec(struct user_page *skel)
+{
+	long page_size = sysconf(_SC_PAGESIZE);
+	long addr = -1;
+
+	if (!alloc_user_pages(skel, 1, &addr))
+		return;
+
+	skel->bss->exec_addr = addr + 1;
+	skel->bss->action = USER_PAGE_EXEC_ONLY;
+	user_page_trigger();
+	ASSERT_EQ(skel->bss->exec_ret, -EINVAL, "misaligned_exec_ret");
+
+	if (addr >= 0)
+		munmap((void *)addr, page_size);
+}
+
+static void test_reject_replacement(struct user_page *skel)
+{
+	long page_size = sysconf(_SC_PAGESIZE);
+	long addr = -1;
+	void *ret;
+
+	if (!alloc_user_pages(skel, 1, &addr))
+		return;
+
+	ret = mmap((void *)addr, page_size, PROT_WRITE,
+		   MAP_PRIVATE | MAP_ANONYMOUS | MAP_FIXED, -1, 0);
+	if (!ASSERT_EQ(ret, (void *)addr, "replace_mmap"))
+		goto out;
+
+	skel->bss->exec_addr = addr;
+	skel->bss->action = USER_PAGE_EXEC_ONLY;
+	user_page_trigger();
+	ASSERT_EQ(skel->bss->exec_ret, -EINVAL, "replacement_not_bpf");
+
+out:
+	if (addr >= 0)
+		munmap((void *)addr, page_size);
+}
+
+static void test_bad_write(struct user_page *skel)
+{
+	skel->bss->action = USER_PAGE_BAD_WRITE;
+	user_page_trigger();
+	ASSERT_EQ(skel->bss->write_ret, -EFAULT, "bad_write_ret");
+}
+
+static void test_unrelated_mapping_exec_rejected(struct user_page *skel)
+{
+	long page_size = sysconf(_SC_PAGESIZE);
+	void *unrelated;
+
+	unrelated = mmap(NULL, page_size, PROT_READ | PROT_WRITE,
+			 MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
+	if (!ASSERT_NEQ(unrelated, MAP_FAILED, "unrelated_mmap"))
+		return;
+
+	skel->bss->exec_addr = (long)unrelated;
+	skel->bss->action = USER_PAGE_EXEC_ONLY;
+	user_page_trigger();
+	ASSERT_EQ(skel->bss->exec_ret, -EINVAL, "bad_exec_ret");
+
+	munmap(unrelated, page_size);
+}
+
+static void test_repeat_exec_rejected(struct user_page *skel)
+{
+	long page_size = sysconf(_SC_PAGESIZE);
+	long addr = -1;
+
+	if (!alloc_user_pages(skel, 1, &addr))
+		return;
+
+	skel->bss->exec_addr = addr;
+	skel->bss->action = USER_PAGE_EXEC_ONLY;
+	user_page_trigger();
+	if (!ASSERT_OK(skel->bss->exec_ret, "first_exec_ret"))
+		goto out;
+	user_page_trigger();
+
+	ASSERT_EQ(skel->bss->exec_ret, -EINVAL, "repeat_exec_ret");
+
+out:
+	if (addr >= 0)
+		munmap((void *)addr, page_size);
+}
+
+static void test_exec_write_protection(struct user_page *skel)
+{
+	long page_size = sysconf(_SC_PAGESIZE);
+	long addr = -1;
+
+	if (!alloc_user_pages(skel, 1, &addr))
+		return;
+
+	skel->bss->exec_addr = addr;
+	skel->bss->action = USER_PAGE_EXEC_ONLY;
+	user_page_trigger();
+	if (!ASSERT_OK(skel->bss->exec_ret, "exec_ret"))
+		goto out;
+
+	errno = 0;
+	if (!ASSERT_EQ(mprotect((void *)addr, page_size,
+				PROT_READ | PROT_WRITE), -1, "restore_write"))
+		goto out;
+
+	ASSERT_EQ(errno, EACCES, "restore_write_errno");
+
+out:
+	if (addr >= 0)
+		munmap((void *)addr, page_size);
+}
+
+static void test_fails(struct user_page *skel)
+{
+	test_invalid_allocation_size(skel);
+	test_misaligned_exec(skel);
+	test_reject_replacement(skel);
+	test_bad_write(skel);
+	test_unrelated_mapping_exec_rejected(skel);
+	test_repeat_exec_rejected(skel);
+	test_exec_write_protection(skel);
+}
+
 void test_user_page(void)
 {
 	struct user_page *skel;
@@ -233,6 +372,8 @@ void test_user_page(void)
 		test_partial_exec(skel);
 	if (test__start_subtest("faultable_write"))
 		test_faultable_write(skel);
+	if (test__start_subtest("fails"))
+		test_fails(skel);
 
 out:
 	user_page__destroy(skel);
tools/testing/selftests/bpf/progs/user_page.c +5 -0
index a855b0b46350..530a0be3737b 100644
@@ -12,6 +12,7 @@ char _license[] SEC("license") = "GPL";
 #define USER_PAGE_EXEC 2
 #define USER_PAGE_EXEC_ONLY 3
 #define USER_PAGE_WRITE 4
+#define USER_PAGE_BAD_WRITE 5
 
 int monitored_pid;
 int action;
@@ -50,6 +51,10 @@ int user_page_test(struct pt_regs *ctx)
 		write_ret = bpf_probe_write_user_faultable((void *)user_addr, data,
 							   sizeof(data));
 		break;
+	case USER_PAGE_BAD_WRITE:
+		write_ret = bpf_probe_write_user_faultable((void *)1, data,
+							    sizeof(data));
+		break;
 	}
 
 	return 0;
Signature · 1 line
2.54.0
#20 lore raw

Re: [RFC bpf-next 9/9] selftests/bpf: Add user page failures test

TL;DR: Commit message claims the test covers replacing sealed mappings, but test_reject_replacement() replaces a still-writable page before any exec call; either reorder the test or reword the…
Quoted 1 line

selftests/bpf: Add user page failures test

TL;DR: Commit message claims the test covers replacing sealed mappings, but test_reject_replacement() replaces a still-writable page before any exec call; either reorder the test or reword the changelog.

Quoted 8 lines

selftests/bpf: Add user page failures test

Cover expected failures for invalid allocation sizes, misaligned or unrelated exec requests, replacement of sealed mappings, repeated execution, bad writes and attempts to restore write permission.

Assisted-by Codex:GPT-6

The commit message lists "replacement of sealed mappings" as a covered failure, but does test_reject_replacement() ever seal the mapping before replacing it?

In this series "seal" refers to the bpf_user_page_exec() transition, which the kernel-doc describes as "The page cannot be made writable again after this call". The test goes straight from alloc_user_pages() to the replacement:

    if (!alloc_user_pages(skel, 1, &addr))
        return;

    ret = mmap((void *)addr, page_size, PROT_WRITE,
               MAP_PRIVATE | MAP_ANONYMOUS | MAP_FIXED, -1, 0);
    if (!ASSERT_EQ(ret, (void *)addr, "replace_mmap"))
        goto out;

    skel->bss->exec_addr = addr;
    skel->bss->action = USER_PAGE_EXEC_ONLY;
    user_page_trigger();
    ASSERT_EQ(skel->bss->exec_ret, -EINVAL, "replacement_not_bpf");

At the time of the MAP_FIXED mmap() the page is still a plain writable VMA, and USER_PAGE_EXEC_ONLY is only triggered afterwards, on the replacement. The replacement itself is also not a failure, since the test asserts that it succeeds. Only the later bpf_user_page_exec() on the new VMA is expected to fail with -EINVAL.

So the scenario the changelog describes (replacing an already exec-only, non-writable page with a writable mapping and checking that exec refuses it) is never run. A reader could also take the changelog and the test name to mean that the kernel refuses to replace sealed mappings, which it does not do.

Should bpf_user_page_exec() be called before the MAP_FIXED mmap() so the test matches the description, or should the changelog be reworded (for example "exec of a replaced BPF-allocated mapping")?

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/39668036298

#21 lore raw
In reply to Jiri Olsa
Quoted 4 lines · Jiri Olsa

hi,
we have few use cases where having the ability to allocate normal or executable user space memory would help (like to replace function in live process or change environment variables)

I think it's better to do what debuggers do:
vm_mmap() with the final prot and access_process_vm(FOLL_FORCE | FOLL_WRITE), like PTRACE_POKETEXT, to fill it. It faults the pages in, writes into r-x private mapping and copy_to_user_page() takes care of icache.
Then no need for bpf_user_page_exec(), do_mprotect_pkey_locked(), the marker, READ_IMPLIES_EXEC special case, get_user_pages(), etc. No mm changes.

Like:

kernel/bpf/helpers.c +76 -0
index 648abe1..ca1d88d 100644
@@ -29,6 +29,7 @@
 #include <linux/task_work.h>
 #include <linux/irq_work.h>
 #include <linux/buildid.h>
+#include <linux/mman.h>
 
 #include "../../lib/kstrtox.h"
 
@@ -3834,6 +3835,60 @@ __bpf_kfunc int bpf_copy_from_user_task_str(void *dst, u32 dst__sz,
 	return ret;
 }
 
+/**
+ * bpf_copy_to_user_task() - Copy data to a task's address space
+ * @unsafe_ptr__ign: Destination address in the task's address space
+ * @src:             Source address, in kernel space
+ * @src__sz:         Number of bytes to copy
+ * @tsk:             The task whose address space will be used
+ * @flags:           Reserved for future use; must be zero
+ *
+ * Faults in the destination and writes the way ptrace(PTRACE_POKETEXT) does:
+ * a private mapping doesn't have to be writable.
+ *
+ * Return: 0 on success, -EINVAL if @flags is non-zero, or -EFAULT if the copy
+ * fails or is partial.
+ */
+__bpf_kfunc int bpf_copy_to_user_task(void __user *unsafe_ptr__ign, const void *src,
+				      u32 src__sz, struct task_struct *tsk, u64 flags)
+{
+	int ret;
+
+	if (unlikely(flags))
+		return -EINVAL;
+
+	if (unlikely(!src__sz))
+		return 0;
+
+	ret = access_process_vm(tsk, (unsigned long)unsafe_ptr__ign, (void *)src,
+				src__sz, FOLL_FORCE | FOLL_WRITE);
+	return ret == src__sz ? 0 : -EFAULT;
+}
+
+/**
+ * bpf_user_mmap() - Map private anonymous memory into the current process
+ * @len:  Length of the mapping
+ * @prot: PROT_* flags of mmap(2)
+ *
+ * Return: The address of the mapping on success, or a negative error code.
+ */
+__bpf_kfunc long bpf_user_mmap(unsigned long len, unsigned long prot)
+{
+	return vm_mmap(NULL, 0, len, prot, MAP_PRIVATE | MAP_ANONYMOUS, 0);
+}
+
+/**
+ * bpf_user_munmap() - Unmap a range of addresses of the current process
+ * @addr: Start of the range
+ * @len:  Length of the range
+ *
+ * Return: 0 on success, or a negative error code.
+ */
+__bpf_kfunc int bpf_user_munmap(unsigned long addr, unsigned long len)
+{
+	return vm_munmap(addr, len);
+}
+
 /* Keep unsigned long in prototype so that kfunc is usable when emitted to
  * vmlinux.h in BPF programs directly, but note that while in BPF prog, the
  * unsigned long always points to 8-byte region on stack, the kernel may only
@@ -5139,6 +5194,25 @@ static const struct btf_kfunc_id_set generic_kfunc_set = {
 	.set   = &generic_btf_ids,
 };
 
+BTF_KFUNCS_START(user_mem_btf_ids)
+BTF_ID_FLAGS(func, bpf_copy_to_user_task, KF_SLEEPABLE | KF_PERFMON)
+BTF_ID_FLAGS(func, bpf_user_mmap, KF_SLEEPABLE | KF_PERFMON)
+BTF_ID_FLAGS(func, bpf_user_munmap, KF_SLEEPABLE | KF_PERFMON)
+BTF_KFUNCS_END(user_mem_btf_ids)
+
+static int user_mem_kfunc_filter(const struct bpf_prog *prog, u32 kfunc_id)
+{
+	if (!btf_id_set8_contains(&user_mem_btf_ids, kfunc_id))
+		return 0;
+	return security_locked_down(LOCKDOWN_BPF_WRITE_USER);
+}
+
+static const struct btf_kfunc_id_set user_mem_kfunc_set = {
+	.owner  = THIS_MODULE,
+	.set    = &user_mem_btf_ids,
+	.filter = user_mem_kfunc_filter,
+};
+
 BTF_ID_LIST(generic_dtor_ids)
 BTF_ID(struct, task_struct)
 BTF_ID(func, bpf_task_release_dtor)
@@ -5266,6 +5340,8 @@ static int __init kfunc_init(void)
 	};
 
 	ret = register_btf_kfunc_id_set(BPF_PROG_TYPE_TRACING, &generic_kfunc_set);
+	/* Sleepable kprobe prog can only be attached to uprobe */
+	ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_KPROBE, &user_mem_kfunc_set);
 	ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_SCHED_CLS, &generic_kfunc_set);
 	ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_XDP, &generic_kfunc_set);
 	ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_STRUCT_OPS, &generic_kfunc_set);

But I'm not at all sure that it's a good idea
to inject code into a process.

pw-bot: cr

#22 lore raw
Quoted 6 lines · Alexei Starovoitov
Quoted 4 lines · Jiri Olsa

hi,
we have few use cases where having the ability to allocate normal or executable user space memory would help (like to replace function in live process or change environment variables)

Quoted 8 lines

I think it's better to do what debuggers do:
vm_mmap() with the final prot and access_process_vm(FOLL_FORCE | FOLL_WRITE), like PTRACE_POKETEXT, to fill it. It faults the pages in, writes into r-x private mapping and copy_to_user_page() takes care of icache.
Then no need for bpf_user_page_exec(), do_mprotect_pkey_locked(), the marker, READ_IMPLIES_EXEC special case, get_user_pages(), etc. No mm changes.

great, much better, will use it in next version, thanks

Quoted 81 lines

Like:

kernel/bpf/helpers.c +56 -0
index 648abe1..ca1d88d 100644
@@ -29,6 +29,7 @@
 #include <linux/task_work.h>
 #include <linux/irq_work.h>
 #include <linux/buildid.h>
+#include <linux/mman.h>
 
 #include "../../lib/kstrtox.h"
 
@@ -3834,6 +3835,60 @@ __bpf_kfunc int bpf_copy_from_user_task_str(void *dst, u32 dst__sz,
 	return ret;
 }
 
+/**
+ * bpf_copy_to_user_task() - Copy data to a task's address space
+ * @unsafe_ptr__ign: Destination address in the task's address space
+ * @src:             Source address, in kernel space
+ * @src__sz:         Number of bytes to copy
+ * @tsk:             The task whose address space will be used
+ * @flags:           Reserved for future use; must be zero
+ *
+ * Faults in the destination and writes the way ptrace(PTRACE_POKETEXT) does:
+ * a private mapping doesn't have to be writable.
+ *
+ * Return: 0 on success, -EINVAL if @flags is non-zero, or -EFAULT if the copy
+ * fails or is partial.
+ */
+__bpf_kfunc int bpf_copy_to_user_task(void __user *unsafe_ptr__ign, const void *src,
+				      u32 src__sz, struct task_struct *tsk, u64 flags)
+{
+	int ret;
+
+	if (unlikely(flags))
+		return -EINVAL;
+
+	if (unlikely(!src__sz))
+		return 0;
+
+	ret = access_process_vm(tsk, (unsigned long)unsafe_ptr__ign, (void *)src,
+				src__sz, FOLL_FORCE | FOLL_WRITE);
+	return ret == src__sz ? 0 : -EFAULT;
+}
+
+/**
+ * bpf_user_mmap() - Map private anonymous memory into the current process
+ * @len:  Length of the mapping
+ * @prot: PROT_* flags of mmap(2)
+ *
+ * Return: The address of the mapping on success, or a negative error code.
+ */
+__bpf_kfunc long bpf_user_mmap(unsigned long len, unsigned long prot)
+{
+	return vm_mmap(NULL, 0, len, prot, MAP_PRIVATE | MAP_ANONYMOUS, 0);
+}
+
+/**
+ * bpf_user_munmap() - Unmap a range of addresses of the current process
+ * @addr: Start of the range
+ * @len:  Length of the range
+ *
+ * Return: 0 on success, or a negative error code.
+ */
+__bpf_kfunc int bpf_user_munmap(unsigned long addr, unsigned long len)
+{
+	return vm_munmap(addr, len);
+}
+
 /* Keep unsigned long in prototype so that kfunc is usable when emitted to
  * vmlinux.h in BPF programs directly, but note that while in BPF prog, the
  * unsigned long always points to 8-byte region on stack, the kernel may only
@@ -5139,6 +5194,25 @@ static const struct btf_kfunc_id_set generic_kfunc_set = {
 	.set   = &generic_btf_ids,
 };
 
+BTF_KFUNCS_START(user_mem_btf_ids)

kernel/bpf/helpers.c static const struct btf_kfunc_id_set generic_kfunc_set = { · line 5207

Quoted 10 lines
kernel/bpf/helpers.c +10 -0
+BTF_ID_FLAGS(func, bpf_copy_to_user_task, KF_SLEEPABLE | KF_PERFMON)
+BTF_ID_FLAGS(func, bpf_user_mmap, KF_SLEEPABLE | KF_PERFMON)
+BTF_ID_FLAGS(func, bpf_user_munmap, KF_SLEEPABLE | KF_PERFMON)
+BTF_KFUNCS_END(user_mem_btf_ids)
+
+static int user_mem_kfunc_filter(const struct bpf_prog *prog, u32 kfunc_id)
+{
+	if (!btf_id_set8_contains(&user_mem_btf_ids, kfunc_id))
+		return 0;
+	return security_locked_down(LOCKDOWN_BPF_WRITE_USER);

ah right.. all of those kfuncs should be behing write_user lockdown

Quoted 21 lines
kernel/bpf/helpers.c +10 -0
+}
+
+static const struct btf_kfunc_id_set user_mem_kfunc_set = {
+	.owner  = THIS_MODULE,
+	.set    = &user_mem_btf_ids,
+	.filter = user_mem_kfunc_filter,
+};
+
 BTF_ID_LIST(generic_dtor_ids)
 BTF_ID(struct, task_struct)
 BTF_ID(func, bpf_task_release_dtor)
@@ -5266,6 +5340,8 @@ static int __init kfunc_init(void)
 	};
 
 	ret = register_btf_kfunc_id_set(BPF_PROG_TYPE_TRACING, &generic_kfunc_set);
+	/* Sleepable kprobe prog can only be attached to uprobe */
+	ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_KPROBE, &user_mem_kfunc_set);
 	ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_SCHED_CLS, &generic_kfunc_set);
 	ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_XDP, &generic_kfunc_set);
 	ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_STRUCT_OPS, &generic_kfunc_set);
Quoted 2 lines

But I'm not at all sure that it's a good idea
to inject code into a process.

well FWIW it's still guarded with the LOCKDOWN_BPF_WRITE_USER same as bpf_probe_write_user helper

at the moment we have fragile way of live patching a userspace process, this would give us a more direct and reliable way for that

thanks,
jirka

#23 lore raw
In reply to Alexei Starovoitov
Quoted 6 lines · Alexei Starovoitov
Quoted 4 lines · Jiri Olsa

hi,
we have few use cases where having the ability to allocate normal or executable user space memory would help (like to replace function in live process or change environment variables)

Quoted 8 lines

I think it's better to do what debuggers do:
vm_mmap() with the final prot and access_process_vm(FOLL_FORCE | FOLL_WRITE), like PTRACE_POKETEXT, to fill it. It faults the pages in, writes into r-x private mapping and copy_to_user_page() takes care of icache.
Then no need for bpf_user_page_exec(), do_mprotect_pkey_locked(), the marker, READ_IMPLIES_EXEC special case, get_user_pages(), etc. No mm changes.

OK thanks that addresses my concerns :)

--
Cheers, Lorenzo