bpf: Add user memory kfuncs
Subsystems (7)
Jiri Olsa
reviewercover bpf: Add user memory kfuncs
hi,
we have few use cases where having the ability to allocate normal or executable user space memory would help (like to replace function in live process or change environment variables).
We also need writes to user memory to handle page faults, while the existing bpf_probe_write_user() helper disables faults during the copy.
This series adds sleepable kfuncs for BPF programs to copy data into the current task’s address space and to create or remove private anonymous mappings with optional execute permissions.
rfc: https://lore.kernel.org/bpf/[email protected]/
v1 changes:
- follow vm_mmap/vm_munmap design suggested by Alexei
- allow user space write only to current process, because we don't have remote task use case and it simplifies security checks
- many changes in tests
thanks,
jirka
---
Jiri Olsa (8):
bpf: Add __srcptr kfunc argument suffix
bpf: Add user memory kfuncs selftests/bpf: Add get_procmap_info helper selftests/bpf: Test __srcptr kfunc arguments selftests/bpf: Test user memory mmap kfuncs selftests/bpf: Test user memory page exec selftests/bpf: Test bpf_copy_to_user kfunc selftests/bpf: Test user memory kfunc errors
Documentation/bpf/kfuncs.rst | 41 +++++++ kernel/bpf/helpers.c | 102 ++++++++++++++++ kernel/bpf/syscall.c | 10 +- kernel/bpf/verifier.c | 20 +++- tools/testing/selftests/bpf/prog_tests/kfunc_call.c | 2 + tools/testing/selftests/bpf/prog_tests/user_mem.c | 338 ++++++++++++++++++++++++++++++++++++++++++++++++++++ tools/testing/selftests/bpf/progs/kfunc_call_fail.c | 10 ++ tools/testing/selftests/bpf/progs/kfunc_call_test.c | 14 +++ tools/testing/selftests/bpf/progs/user_mem.c | 111 +++++++++++++++++ tools/testing/selftests/bpf/test_kmods/bpf_testmod.c | 10 ++ tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h | 2 + tools/testing/selftests/bpf/trace_helpers.c | 38 +++++- tools/testing/selftests/bpf/trace_helpers.h | 1 + 13 files changed, 686 insertions(+), 13 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/user_mem.c create mode 100644 tools/testing/selftests/bpf/progs/user_mem.c
Jiri Olsa
reviewerPATCH 1/8 bpf: Add __srcptr kfunc argument suffix
Generic sized pointer arguments are treated as writable memory. This prevents kfuncs that only read from a source buffer from accepting read-only BPF memory, such as const data.
Adding __srcptr to mark pointers the kfunc only reads.
The get_kfunc_arg_type change sets the initial memory access mode and the resolve_func_arg_type preserves the suffix’s access mode when it converts a BTF pointer classification into a memory argument.
--- Documentation/bpf/kfuncs.rst | 24 ++++++++++++++++++++++++ kernel/bpf/verifier.c | 20 +++++++++++++++++--- 2 files changed, 41 insertions(+), 3 deletions(-)
Documentation/bpf/kfuncs.rst +24 -0
@@ -351,6 +351,30 @@ In this case, a program attached to the ``inode_init_security`` LSM hook can pass the hook's own ``xattr_count`` argument through to the kfunc, which claims xattr slots by writing through it. +2.3.10 __srcptr Annotation +-------------------------- + +This annotation marks a pointer argument as a source buffer that the kfunc +reads without writing. The verifier requires read access, so the source may +point to read-only or read/write memory. Pair variable-size buffers with a +following ``__sz`` or ``__szk`` argument. For a pointer to a scalar or to a +struct of scalars, the verifier uses the pointee's BTF size; a ``void *`` +requires an explicit size. + +For example, the first kfunc takes a variable-size buffer, while the second +takes a scalar source:: + + __bpf_kfunc void bpf_consume_data(const void *src__srcptr, + u32 src__sz) + { + ... + } + + __bpf_kfunc void bpf_consume_scalar(const u32 *src__srcptr) + { + ... + } + .. _BPF_kfunc_nodef: 2.4 Using an existing kernel function
kernel/bpf/verifier.c +17 -3
@@ -13205,6 +13205,20 @@ static bool is_kfunc_arg_mem_size(const struct btf *btf,return btf_param_match_suffix (btf, arg,"__sz" ); } +static bool is_kfunc_arg_srcptr (const struct btf *btf, +const struct btf_param *arg) +{ +return btf_param_match_suffix (btf, arg,"__srcptr" ); +} + +static int mem_prot (const struct btf *btf,const struct btf_param *arg) +{ +if (is_kfunc_arg_srcptr (btf, arg)) +return MEM_RDONLY ; +else +return MEM_WRITE ; +} +static bool is_kfunc_arg_const_mem_size (const struct btf *btf,const struct btf_param *arg) {
@@ -13585,7 +13599,7 @@ static int resolve_func_arg_type(struct bpf_verifier_env *env, PTR_ERR(resolve_ret));return -EINVAL ; } - *arg_type =ARG_PTR_TO_MEM |MEM_FIXED_SIZE |MEM_WRITE | + *arg_type =ARG_PTR_TO_MEM |MEM_FIXED_SIZE |mem_prot (meta->btf, &args[arg]) | (*arg_type & (PTR_MAYBE_NULL |MEM_UNINIT ));return 0 ;
@@ -14137,7 +14151,7 @@ get_kfunc_arg_type(struct bpf_verifier_env *env, struct bpf_call_arg_meta *meta, reg_arg_name(env, argno), btf_type_str(ref_t), ref_tname);return -EINVAL ; } - arg_type =ARG_PTR_TO_MEM |MEM_WRITE ; + arg_type =ARG_PTR_TO_MEM |mem_prot (meta->btf, &args[arg]); }else if (btf_type_is_struct (ref_t))/* A pointer to a struct without a size argument is classified as ARG_PTR_TO_BTF_ID */ arg_type =ARG_PTR_TO_BTF_ID ;
@@ -14162,7 +14176,7 @@ get_kfunc_arg_type(struct bpf_verifier_env *env, struct bpf_call_arg_meta *meta,return -EINVAL ; } proto->arg_size[arg] = type_size; - arg_type =ARG_PTR_TO_MEM |MEM_FIXED_SIZE |MEM_WRITE ; + arg_type =ARG_PTR_TO_MEM |MEM_FIXED_SIZE |mem_prot (meta->btf, &args[arg]); }if (is_kfunc_arg_uninit (meta->btf, &args[arg]))
Signature · 1 line
2.54.0
Re: [PATCH bpf-next 1/8] bpf: Add __srcptr kfunc argument suffix
Quoted 6 lines · Jiri Olsa
Generic sized pointer arguments are treated as writable memory. This prevents kfuncs that only read from a source buffer from accepting read-only BPF memory, such as const data.
Adding
__srcptrto mark pointers the kfunc only reads.
why can't we infer read-only from constness of a pointer? what's the advantage of __srcptr (which, honestly, says absolutely nothing, "__readonly" or "__ro" would probably make more sense if we have to go with suffix)
Quoted 12 lines
The
get_kfunc_arg_typechange sets the initial memory access mode and theresolve_func_arg_typepreserves the suffix’s access mode when it converts a BTF pointer classification into a memory argument.Assisted-by Codex:GPT-6Signed-off-by Jiri Olsa <[email protected]>--- Documentation/bpf/kfuncs.rst | 24 ++++++++++++++++++++++++ kernel/bpf/verifier.c | 20 +++++++++++++++++--- 2 files changed, 41 insertions(+), 3 deletions(-)
[...]
Re: [PATCH bpf-next 1/8] bpf: Add __srcptr kfunc argument suffix
Quoted 6 lines · Andrii Nakryiko
Quoted 5 lines · Jiri Olsa
Generic sized pointer arguments are treated as writable memory. This prevents kfuncs that only read from a source buffer from accepting read-only BPF memory, such as const data.
Quoted 6 lines
Quoted 1 line
Adding
__srcptrto mark pointers the kfunc only reads.why can't we infer read-only from constness of a pointer? what's the advantage of
__srcptr(which, honestly, says absolutely nothing, "__readonly" or "__ro" would probably make more sense if we have to go with suffix)
the reason was that current pointer arguments are marked as MEM_WRITE, so passing readonly pointer to bpf_copy_to_user caused verifier error
but I think we can use const, did not think of that.. will change
thanks,
jirka
Quoted 15 lines
Quoted 13 lines
The
get_kfunc_arg_typechange sets the initial memory access mode and theresolve_func_arg_typepreserves the suffix’s access mode when it converts a BTF pointer classification into a memory argument.Assisted-by Codex:GPT-6Signed-off-by Jiri Olsa <[email protected]>--- Documentation/bpf/kfuncs.rst | 24 ++++++++++++++++++++++++ kernel/bpf/verifier.c | 20 +++++++++++++++++--- 2 files changed, 41 insertions(+), 3 deletions(-)[...]
Re: [PATCH bpf-next 1/8] bpf: Add __srcptr kfunc argument suffix
In reply to Jiri OlsaQuoted 5 lines · Jiri Olsa
Generic sized pointer arguments are treated as writable memory. This prevents kfuncs that only read from a source buffer from accepting read-only BPF memory, such as const data.
Adding
__srcptrto mark pointers the kfunc only reads.
+1 to Andrii's point re: const.
An arg name has one suffix, so far, so __srcptr cannot mark 'const union bpf_ksock_addr *addr__nullable' of bpf_ksock_sendto().
Tentative patch:
Documentation/bpf/kfuncs.rst +4 -0
@@ -137,6 +137,10 @@ argument as its size. By default, without __sz annotation, the size of the type of the pointer is used. Without __sz annotation, a kfunc cannot accept a void pointer. +The verifier checks such memory for read and write access. When the argument +is a pointer to const, like ``const void *mem``, the verifier checks it for +read access only, and read-only memory (e.g. ``.rodata``) can be passed. + 2.3.2 __k Annotation --------------------
kernel/bpf/verifier.c +21 -3
@@ -13217,6 +13217,22 @@ static bool is_kfunc_arg_const_mem_size(const struct btf *btf,return btf_param_match_suffix (btf, arg,"__szk" ); } +/* A kfunc only reads through 'const T *arg'. The compiler enforces it. */ +static enum bpf_type_flag kfunc_arg_mem_flag (const struct btf *btf, +const struct btf_param *arg) +{ +const struct btf_type *t =btf_type_skip_modifiers (btf, arg->type,NULL ); +u32 id, ref_id; + +btf_type_skip_modifiers (btf, t->type, &ref_id); +for (id = t->type; id != ref_id; id = t->type) { + t =btf_type_by_id (btf, id); +if (btf_kind (t) ==BTF_KIND_CONST ) +return MEM_RDONLY ; + } +return MEM_WRITE ; +} +static bool is_kfunc_arg_constant (const struct btf *btf,const struct btf_param *arg) {return btf_param_match_suffix (btf, arg,"__k" );
@@ -13585,7 +13601,8 @@ static int resolve_func_arg_type(struct bpf_verifier_env *env, PTR_ERR(resolve_ret));return -EINVAL ; } - *arg_type =ARG_PTR_TO_MEM |MEM_FIXED_SIZE |MEM_WRITE | + *arg_type =ARG_PTR_TO_MEM |MEM_FIXED_SIZE | +kfunc_arg_mem_flag (meta->btf, &args[arg]) | (*arg_type & (PTR_MAYBE_NULL |MEM_UNINIT ));return 0 ;
@@ -14137,7 +14154,7 @@ get_kfunc_arg_type(struct bpf_verifier_env *env, struct bpf_call_arg_meta *meta, reg_arg_name(env, argno), btf_type_str(ref_t), ref_tname);return -EINVAL ; } - arg_type =ARG_PTR_TO_MEM |MEM_WRITE ; + arg_type =ARG_PTR_TO_MEM |kfunc_arg_mem_flag (meta->btf, &args[arg]); }else if (btf_type_is_struct (ref_t))/* A pointer to a struct without a size argument is classified as ARG_PTR_TO_BTF_ID */ arg_type =ARG_PTR_TO_BTF_ID ;
@@ -14162,7 +14179,8 @@ get_kfunc_arg_type(struct bpf_verifier_env *env, struct bpf_call_arg_meta *meta,return -EINVAL ; } proto->arg_size[arg] = type_size; - arg_type =ARG_PTR_TO_MEM |MEM_FIXED_SIZE |MEM_WRITE ; + arg_type =ARG_PTR_TO_MEM |MEM_FIXED_SIZE | +kfunc_arg_mem_flag (meta->btf, &args[arg]); }if (is_kfunc_arg_uninit (meta->btf, &args[arg]))
pw-bot: cr
Jiri Olsa
reviewerPATCH 2/8 bpf: Add user memory kfuncs
In reply to Jiri OlsaAdd sleepable kfuncs to copy data into a task's address space and to map or unmap private anonymous memory in the current process.
Require a usable user mm for mmap and munmap. Register the kfuncs for KPROBE programs and gate them on CAP_SYS_ADMIN BPF token capability and the BPF_WRITE_USER lockdown check.
Use FOLL_FORCE | FOLL_WRITE for task memory copies and return -EFAULT when a copy is incomplete.
Rename the syscall info-copy helper to bpf_copy_to_user_buf to avoid the name collision.
--- kernel/bpf/helpers.c | 102 +++++++++++++++++++++++++++++++++++++++++++ kernel/bpf/syscall.c | 10 ++--- 2 files changed, 107 insertions(+), 5 deletions(-)
kernel/bpf/helpers.c +102 -0
@@ -29,6 +29,9 @@#include <linux/task_work.h> #include <linux/irq_work.h> #include <linux/buildid.h> +#include <linux/mm.h> +#include <linux/mman.h> +#include <linux/limits.h> #include "../../lib/kstrtox.h"
@@ -3738,6 +3741,83 @@ __bpf_kfunc int bpf_copy_from_user_mm(void *dst, u32 dst__sz,return ret; } +/** +* bpf_copy_to_user() - Copy data to the current task's address space +* @unsafe_ptr__ign: Destination address in the current task's address space +* @src__srcptr: Source address, in kernel space +* @src__sz: Number of bytes to copy +* @flags: Reserved for future use; must be zero +* +* Faults in the destination and writes the way ptrace(PTRACE_POKETEXT) does: +* a private mapping doesn't have to be writable. A partial copy is not undone, +* the bytes before the fault stay written. +* +* The caller must coordinate with other threads so they do not execute the +* destination range while it is being modified, and ensure execution begins +* only after the copy completes. +* +* Return: 0 on success, -EINVAL if @flags is non-zero, or -EFAULT if the copy +* fails or is partial. +*/ +__bpf_kfunc int bpf_copy_to_user(void __user *unsafe_ptr__ign, const void *src__srcptr, +u32 src__sz,u64 flags) +{ +int ret; + +if (unlikely (flags)) +return -EINVAL ; +if (unlikely (src__sz >S32_MAX )) +return -EINVAL ; +if (unlikely (!src__sz)) +return 0 ; + ret =access_process_vm (current, (unsigned long )unsafe_ptr__ign, (void *)src__srcptr, + src__sz,FOLL_FORCE |FOLL_WRITE ); +return ret == src__sz ?0 : -EFAULT ; +} + +/* bpf_wq and bpf_task_work callbacks can run without a usable mm. */ +static bool bpf_user_mm_ok (void ) +{ +return current->mm && !(current->flags & (PF_KTHREAD |PF_EXITING )); +} + +/** +* bpf_user_mmap() - Map private anonymous memory into the current process +* @len: Length of the mapping +* @prot: PROT_* flags of mmap(2), including PROT_EXEC +* +* This maps memory by calling vm_mmap() directly. It does not invoke the +* mmap(2) syscall, so seccomp filters do not mediate this mapping. +* +* Return: The address of the mapping on success, -EPERM if the caller has no +* user mm, or an unsigned-long encoded negative error code, as returned by +* vm_mmap(). +*/ +__bpf_kfunc unsigned long bpf_user_mmap (unsigned long len,unsigned long prot) +{ +if (!bpf_user_mm_ok ()) +return -EPERM ; +return vm_mmap (NULL ,0 , len, prot,MAP_PRIVATE |MAP_ANONYMOUS ,0 ); +} + +/** +* bpf_user_munmap() - Unmap a range of addresses of the current process +* @addr: Start of the range +* @len: Length of the range +* +* This unmaps memory by calling vm_munmap() directly. It does not invoke the +* munmap(2) syscall, so seccomp filters do not mediate this unmapping. +* +* Return: 0 on success, -EPERM if the caller has no user mm, or a negative +* error code. +*/ +__bpf_kfunc int bpf_user_munmap(unsigned long addr, unsigned long len) +{ +if (!bpf_user_mm_ok ()) +return -EPERM ; +return vm_munmap (addr, len); +} +/** * bpf_copy_from_user_mm_str() - Copy a string from an address space * @dst: Destination address, in kernel space. This buffer must be
@@ -5139,6 +5219,27 @@ static const struct btf_kfunc_id_set generic_kfunc_set = { .set = &generic_btf_ids, }; +BTF_KFUNCS_START(user_mem_btf_ids) +BTF_ID_FLAGS(func, bpf_copy_to_user, KF_SLEEPABLE | KF_PERFMON) +BTF_ID_FLAGS(func, bpf_user_mmap, KF_SLEEPABLE | KF_PERFMON) +BTF_ID_FLAGS(func, bpf_user_munmap, KF_SLEEPABLE | KF_PERFMON) +BTF_KFUNCS_END(user_mem_btf_ids) + +static int user_mem_kfunc_filter(const struct bpf_prog *prog, u32 kfunc_id) +{ + if (!btf_id_set8_contains(&user_mem_btf_ids, kfunc_id)) + return 0; + if (!bpf_token_capable(prog->aux->token, CAP_SYS_ADMIN)) + return -EPERM; + return security_locked_down(LOCKDOWN_BPF_WRITE_USER); +} + +static const struct btf_kfunc_id_set user_mem_kfunc_set = { + .owner = THIS_MODULE, + .set = &user_mem_btf_ids, + .filter = user_mem_kfunc_filter, +}; + BTF_ID_LIST(generic_dtor_ids) BTF_ID(struct, task_struct) BTF_ID(func, bpf_task_release_dtor)
@@ -5266,6 +5367,7 @@ static int __init kfunc_init(void) }; ret =register_btf_kfunc_id_set (BPF_PROG_TYPE_TRACING , &generic_kfunc_set); + ret = ret ?:register_btf_kfunc_id_set (BPF_PROG_TYPE_KPROBE , &user_mem_kfunc_set); ret = ret ?:register_btf_kfunc_id_set (BPF_PROG_TYPE_SCHED_CLS , &generic_kfunc_set); ret = ret ?:register_btf_kfunc_id_set (BPF_PROG_TYPE_XDP , &generic_kfunc_set); ret = ret ?:register_btf_kfunc_id_set (BPF_PROG_TYPE_STRUCT_OPS , &generic_kfunc_set);
kernel/bpf/syscall.c +5 -5
@@ -3936,8 +3936,8 @@ static void bpf_raw_tp_link_show_fdinfo(const struct bpf_link *link, raw_tp_link->cookie); } -static int bpf_copy_to_user (char __user *ubuf,const char *buf,u32 ulen, -u32 len) +static int bpf_copy_to_user_buf (char __user *ubuf,const char *buf,u32 ulen, +u32 len) {if (ulen >= len +1 ) {if (copy_to_user (ubuf, buf, len +1 ))
@@ -3974,7 +3974,7 @@ static int bpf_raw_tp_link_fill_link_info(const struct bpf_link *link,if (!ubuf)return 0 ; -return bpf_copy_to_user (ubuf, tp_name, ulen, tp_len); +return bpf_copy_to_user_buf (ubuf, tp_name, ulen, tp_len); }static const struct bpf_link_ops bpf_raw_tp_link_lops = {
@@ -4035,7 +4035,7 @@ static int bpf_perf_link_fill_common(const struct perf_event *event, goto out;if (buf) { - err =bpf_copy_to_user (uname, buf, ulen, len); + err =bpf_copy_to_user_buf (uname, buf, ulen, len); }else {char zero ='\0' ;
@@ -5714,7 +5714,7 @@ static int bpf_task_fd_query_copy(const union bpf_attr *attr,if (put_user (zero, ubuf))return -EFAULT ; }else { - err =bpf_copy_to_user (ubuf, buf, input_len, len); + err =bpf_copy_to_user_buf (ubuf, buf, input_len, len);if (err == -EFAULT )return err; }
Signature · 1 line
2.54.0
Quoted 21 lines · Jiri Olsa
Add sleepable kfuncs to copy data into a task's address space and to map or unmap private anonymous memory in the current process.
Require a usable user mm for mmap and munmap. Register the kfuncs for KPROBE programs and gate them on
CAP_SYS_ADMINBPF token capability and theBPF_WRITE_USERlockdown check.Use
FOLL_FORCE|FOLL_WRITEfor task memory copies and return-EFAULTwhen a copy is incomplete.Rename the syscall info-copy helper to
bpf_copy_to_user_bufto avoid the name collision.Assisted-by Codex:GPT-6Signed-off-by Jiri Olsa <[email protected]>--- kernel/bpf/helpers.c | 102 +++++++++++++++++++++++++++++++++++++++++++ kernel/bpf/syscall.c | 10 ++--- 2 files changed, 107 insertions(+), 5 deletions(-)
[...]
kernel/bpf/helpers.c __bpf_kfunc int bpf_copy_from_user_mm(void *dst, u32 dst__sz, · line 3796
Quoted 13 lines
kernel/bpf/helpers.c +13
-0+/** +* bpf_user_mmap() - Map private anonymous memory into the current process +* @len: Length of the mapping +* @prot: PROT_* flags of mmap(2), including PROT_EXEC +* +* This maps memory by calling vm_mmap() directly. It does not invoke the +* mmap(2) syscall, so seccomp filters do not mediate this mapping. +* +* Return: The address of the mapping on success, -EPERM if the caller has no +* user mm, or an unsigned-long encoded negative error code, as returned by +* vm_mmap(). +*/ +__bpf_kfunc unsigned long bpf_user_mmap (unsigned long len,unsigned long prot)
bpf_user_anon_mmap(), so we can conceivably have a more generic bpf_user_mmap(struct file*, ...) in the future?
Quoted 6 lines
kernel/bpf/helpers.c +6
-0+{ +if (!bpf_user_mm_ok ()) +return -EPERM ; +return vm_mmap (NULL ,0 , len, prot,MAP_PRIVATE |MAP_ANONYMOUS ,0 ); +} +
[...]
Quoted 25 lines · Andrii Nakryiko
Quoted 21 lines · Jiri Olsa
Add sleepable kfuncs to copy data into a task's address space and to map or unmap private anonymous memory in the current process.
Require a usable user mm for mmap and munmap. Register the kfuncs for KPROBE programs and gate them on
CAP_SYS_ADMINBPF token capability and theBPF_WRITE_USERlockdown check.Use
FOLL_FORCE|FOLL_WRITEfor task memory copies and return-EFAULTwhen a copy is incomplete.Rename the syscall info-copy helper to
bpf_copy_to_user_bufto avoid the name collision.Assisted-by Codex:GPT-6Signed-off-by Jiri Olsa <[email protected]>--- kernel/bpf/helpers.c | 102 +++++++++++++++++++++++++++++++++++++++++++ kernel/bpf/syscall.c | 10 ++--- 2 files changed, 107 insertions(+), 5 deletions(-)[...]
Quoted 16 lines
kernel/bpf/helpers.c __bpf_kfunc int bpf_copy_from_user_mm(void *dst, u32 dst__sz, · line 3796
Quoted 13 lines
kernel/bpf/helpers.c +13
-0+/** +* bpf_user_mmap() - Map private anonymous memory into the current process +* @len: Length of the mapping +* @prot: PROT_* flags of mmap(2), including PROT_EXEC +* +* This maps memory by calling vm_mmap() directly. It does not invoke the +* mmap(2) syscall, so seccomp filters do not mediate this mapping. +* +* Return: The address of the mapping on success, -EPERM if the caller has no +* user mm, or an unsigned-long encoded negative error code, as returned by +* vm_mmap(). +*/ +__bpf_kfunc unsigned long bpf_user_mmap (unsigned long len,unsigned long prot)
bpf_user_anon_mmap(), so we can conceivably have a more genericbpf_user_mmap(struct file*, ...)in the future?
yes, will change
thanks,
jirka
[email protected]
AI reviewTL;DR: - bpf_user_mmap() may return -ENOMEM for ia32 tasks (and some 64-bit tasks with X32_ABI) on x86, since vm_mmap() isn't called from a syscall - Skipping arch_validate_prot() may allow mappings…
Quoted 1 line
bpf: Add user memory kfuncs
TL;DR:
- bpf_user_mmap() may return -ENOMEM for ia32 tasks (and some 64-bit tasks with X32_ABI) on x86, since vm_mmap() isn't called from a syscall
- Skipping arch_validate_prot() may allow mappings mmap(2) rejects, e.g. powerpc PROT_SAO
Quoted 4 lines
kernel/bpf/helpers.c +0
-0
[ ... ]
Quoted 1 line
kernel/bpf/helpers.c +0
-0@@ -3738,6 +3741,83 @@ __bpf_kfunc int bpf_copy_from_user_mm(void *dst, u32 dst__sz,
[ ... ]
Quoted 8 lines
kernel/bpf/helpers.c +8
-0+/** +* bpf_user_mmap() - Map private anonymous memory into the current process +* @len: Length of the mapping +* @prot: PROT_* flags of mmap(2), including PROT_EXEC +* +* This maps memory by calling vm_mmap() directly. It does not invoke the +* mmap(2) syscall, so seccomp filters do not mediate this mapping. +*
kernel/bpf/helpers.c __bpf_kfunc int bpf_copy_from_user_mm(void *dst, u32 dst__sz, · line 3801
Quoted 10 lines
kernel/bpf/helpers.c +10
-0+* Return: The address of the mapping on success, -EPERM if the caller has no +* user mm, or an unsigned-long encoded negative error code, as returned by +* vm_mmap(). +*/ +__bpf_kfunc unsigned long bpf_user_mmap (unsigned long len,unsigned long prot) +{ +if (!bpf_user_mm_ok ()) +return -EPERM ; +return vm_mmap (NULL ,0 , len, prot,MAP_PRIVATE |MAP_ANONYMOUS ,0 ); +}
Can this vm_mmap() call fail for ia32 tasks, and for 64-bit tasks when CONFIG_X86_X32_ABI=y, because it is not made from a syscall?
On x86_64 the address picked by vm_mmap() depends on the bitness of the current syscall:
vm_mmap()
do_mmap()
__get_unmapped_area()
arch_get_unmapped_area_topdown()
info.high_limit = get_mmap_base(0);arch/x86/mm/mmap.c:get_mmap_base() {
if (in_32bit_syscall()) {
return is_legacy ? mm->mmap_compat_legacy_base
: mm->mmap_compat_base;
}
return is_legacy ? mm->mmap_legacy_base : mm->mmap_base;
}The sleepable kprobe programs that can call this kfunc run either from uprobe_notify_resume() -> handle_swbp() -> handler_chain() (or uprobe_prog_run()) on the int3 return-to-user path, or from bpf_task_work callbacks run by task_work_run(). TS_COMPAT is set only in syscall_32_enter() and is cleared on every return to user mode, so it is clear in all of these paths.
For an ia32 task (TIF_ADDR32) on a kernel without CONFIG_X86_X32_ABI, the top-down search therefore runs below the 64-bit mm->mmap_base. Everything above 4GB is free in such a task, so the search returns an address above 4GB and __get_unmapped_area() rejects it:
if (addr > TASK_SIZE - len)
return -ENOMEM;
The bottom-up legacy layout fails the same way, because find_start_end() also starts from get_mmap_base(1). So the kfunc always returns -ENOMEM for ia32 processes.
With CONFIG_X86_X32_ABI=y the result is the opposite. Exception entry stores orig_ax = -1, so in_x32_syscall() is true in the uprobe handler and 64-bit processes get the compat (below 4GB) layout. A large len then fails with -ENOMEM in the bottom-up fallback (end = task_size_32bit()) even though 64-bit address space is free.
The kernel documents this dependency in __set_personality_x32(): "The x86 mmap() code relies on the syscall bitness so set x32 syscall bit right here to make in_32bit_syscall() work during exec()". The uprobes XOL area avoids the problem by passing an explicit TASK_SIZE - PAGE_SIZE hint.
Neither the kernel-doc ("into the current process") nor the commit message mentions this limitation, and the selftests later in the series only exercise 64-bit binaries. Should this limitation be handled or documented?
Also, does passing prot straight to vm_mmap() skip the architecture prot validation that mmap(2) performs? The kernel-doc describes @prot as "PROT_* flags of mmap(2)", but the mmap(2) entry points call arch_validate_prot() and the vm_mmap() path does not. For example, on powerpc64:
arch/powerpc/kernel/syscalls.c:do_mmap2() {
if (!arch_validate_prot(prot, addr))
return -EINVAL;
...
}arch_validate_prot() in arch/powerpc/include/asm/mman.h rejects:
- any bit outside PROT_READ|PROT_WRITE|PROT_EXEC|PROT_SEM|PROT_SAO
- PROT_SAO when !cpu_has_feature(CPU_FTR_SAO), for example on Power10
- PROT_SAO in a guest without CONFIG_PPC_PROT_SAO_LPAR
vm_mmap() -> vm_mmap_pgoff() -> do_mmap() never calls arch_validate_prot(). Instead do_mmap() -> calc_vm_prot_bits() -> arch_calc_vm_prot_bits() sets VM_SAO for PROT_SAO without any check, and vm_get_page_prot() then adds _PAGE_SAO to the page table entries.
So a program can create strong-access-ordering mappings that the architecture code deliberately refuses. The Kconfig help for PPC_PROT_SAO_LPAR warns that using them may cause issues when migrating a guest from a host that supports strong access ordering to one that does not.
Would it make sense to validate prot with arch_validate_prot(), or to restrict it to PROT_READ|PROT_WRITE|PROT_EXEC?
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/40959191771
On Thu, Oct 29, 2026 at 10:12:12PM +0000, [email protected] wrote:
SNIP
Quoted 11 lines
arch_validate_prot()inarch/powerpc/include/asm/mman.hrejects:- any bit outside PROT_READ|PROT_WRITE|PROT_EXEC|PROT_SEM|PROT_SAO
-PROT_SAOwhen !cpu_has_feature(CPU_FTR_SAO), for example on Power10
-PROT_SAOin a guest withoutCONFIG_PPC_PROT_SAO_LPAR
vm_mmap()->vm_mmap_pgoff()->do_mmap()never callsarch_validate_prot(). Insteaddo_mmap()->calc_vm_prot_bits()->arch_calc_vm_prot_bits()setsVM_SAOforPROT_SAOwithout any check, andvm_get_page_prot()then adds_PAGE_SAOto the page table entries.
Quoted 7 lines
So a program can create strong-access-ordering mappings that the architecture code deliberately refuses. The
Kconfighelp forPPC_PROT_SAO_LPARwarns that using them may cause issues when migrating a guest from a host that supports strong access ordering to one that does not.Would it make sense to validate prot with
arch_validate_prot(), or to restrict it to PROT_READ|PROT_WRITE|PROT_EXEC?
arch_validate_prot seems ok to me
jirka
On Thu, Oct 29, 2026 at 10:12:12PM +0000, [email protected] wrote:
SNIP
Quoted 52 lines
Review
Quoted 6 lines
(quoted diff) +6
-0+__bpf_kfunc unsigned long bpf_user_mmap(unsigned long len, unsigned long prot) +{ + if (!bpf_user_mm_ok()) + return -EPERM; + return vm_mmap(NULL, 0, len, prot, MAP_PRIVATE | MAP_ANONYMOUS, 0); +}Can this
vm_mmap()call fail for ia32 tasks, and for 64-bit tasks whenCONFIG_X86_X32_ABI=y, because it is not made from a syscall?On
x86_64the address picked byvm_mmap()depends on the bitness of the current syscall:
vm_mmap()
do_mmap()__get_unmapped_area() arch_get_unmapped_area_topdown() info.high_limit = get_mmap_base(0);arch/x86/mm/mmap.c:get_mmap_base() { if (in_32bit_syscall()) { return is_legacy ? mm->mmap_compat_legacy_base : mm->mmap_compat_base; } return is_legacy ? mm->mmap_legacy_base : mm->mmap_base; }The sleepable kprobe programs that can call this kfunc run either from
uprobe_notify_resume()->handle_swbp()->handler_chain()(oruprobe_prog_run()) on the int3 return-to-user path, or frombpf_task_workcallbacks run bytask_work_run().TS_COMPATis set only insyscall_32_enter()and is cleared on every return to user mode, so it is clear in all of these paths.For an ia32 task (
TIF_ADDR32) on a kernel withoutCONFIG_X86_X32_ABI, the top-down search therefore runs below the 64-bitmm->mmap_base. Everything above 4GB is free in such a task, so the search returns an address above 4GB and__get_unmapped_area()rejects it:if (addr >
TASK_SIZE- len)
return-ENOMEM;The bottom-up legacy layout fails the same way, because
find_start_end()also starts fromget_mmap_base(1). So the kfunc always returns-ENOMEMfor ia32 processes.With
CONFIG_X86_X32_ABI=ythe result is the opposite. Exception entry storesorig_ax= -1, soin_x32_syscall()is true in the uprobe handler and 64-bit processes get the compat (below 4GB) layout. A large len then fails with-ENOMEMin the bottom-up fallback (end =task_size_32bit()) even though 64-bit address space is free.
Quoted 8 lines
The kernel documents this dependency in
__set_personality_x32(): "The x86mmap()code relies on the syscall bitness so set x32 syscall bit right here to makein_32bit_syscall()work duringexec()". The uprobes XOL area avoids the problem by passing an explicitTASK_SIZE-PAGE_SIZEhint.Neither the kernel-doc ("into the current process") nor the commit message mentions this limitation, and the selftests later in the series only exercise 64-bit binaries. Should this limitation be handled or documented?
that might be a problem.. we could use similar workaround that's used in arch_uprobe_get_xol_area and set TS_COMPAT before we call vm_mmap and restore after (like below)
or we can always bail out for compat apps
jirka
---
kernel/bpf/helpers.c +25 -1
@@ -32,6 +32,9 @@#include <linux/mm.h> #include <linux/mman.h> #include <linux/limits.h> +#ifdef CONFIG_X86_64 +#include <asm/thread_info.h> +#endif #include "../../lib/kstrtox.h"
@@ -3781,6 +3784,27 @@ static bool bpf_user_mm_ok(void)return current->mm && !(current->flags & (PF_KTHREAD |PF_EXITING )); } +#ifdef CONFIG_X86_64 +static unsigned long bpf_user_vm_mmap (unsigned long len,unsigned long prot) +{ +struct thread_info *ti =current_thread_info (); +bool compat =test_thread_flag (TIF_ADDR32 ); +unsigned long addr; + +if (compat) + ti->status |=TS_COMPAT ; + addr =vm_mmap (NULL ,0 , len, prot,MAP_PRIVATE |MAP_ANONYMOUS ,0 ); +if (compat) + ti->status &= ~TS_COMPAT ; +return addr; +} +#else +static unsigned long bpf_user_vm_mmap (unsigned long len,unsigned long prot) +{ +return vm_mmap (NULL ,0 , len, prot,MAP_PRIVATE |MAP_ANONYMOUS ,0 ); +} +#endif +/** * bpf_user_mmap() - Map private anonymous memory into the current process * @len: Length of the mapping
@@ -3797,7 +3821,7 @@ __bpf_kfunc unsigned long bpf_user_mmap(unsigned long len, unsigned long prot) {if (!bpf_user_mm_ok ())return -EPERM ; -return vm_mmap (NULL ,0 , len, prot,MAP_PRIVATE |MAP_ANONYMOUS ,0 ); +return bpf_user_vm_mmap (len, prot); }/**
Jiri Olsa
reviewerPATCH 3/8 selftests/bpf: Add get_procmap_info helper
In reply to Jiri OlsaAdding get_procmap_info that queries the current process VMA range and permissions through PROCMAP_QUERY. Extend procmap_query to return the VMA end address and update its callers.
I don't think we need the search fallback, the PROCMAP api is merged for some time now.
--- tools/testing/selftests/bpf/trace_helpers.c | 38 ++++++++++++++++++--- tools/testing/selftests/bpf/trace_helpers.h | 1 + 2 files changed, 34 insertions(+), 5 deletions(-)
tools/testing/selftests/bpf/trace_helpers.c +33 -5
@@ -286,7 +286,8 @@ unsigned long long module_get_base_addr_sec(const char *module, const char *sec)#ifdef PROCMAP_QUERY int env_verbosity__weak =0 ; -static int procmap_query (int fd,const void *addr,__u32 query_flags,size_t *start,size_t *offset,int *flags) +static int procmap_query (int fd,const void *addr,__u32 query_flags, +size_t *start,size_t *end,size_t *offset,int *flags) {char path_buf[PATH_MAX ], build_id_buf[20 ];struct procmap_query q;
@@ -325,6 +326,7 @@ static int procmap_query(int fd, const void *addr, __u32 query_flags, size_t *st } *start = q.vma_start; + *end = q.vma_end; *offset = q.vma_offset; *flags = q.vma_flags;return 0 ;
@@ -333,13 +335,37 @@ static int procmap_query(int fd, const void *addr, __u32 query_flags, size_t *st# ifndef PROCMAP_QUERY_VMA_EXECUTABLE # define PROCMAP_QUERY_VMA_EXECUTABLE 0x04# endif - -static int procmap_query (int fd,const void *addr,__u32 query_flags,size_t *start,size_t *offset,int *flags) +static int procmap_query (int fd,const void *addr,__u32 query_flags, +size_t *start,size_t *end,size_t *offset,int *flags) {return -EOPNOTSUPP ; }#endif +int get_procmap_info (const void *addr,size_t *start,size_t *end,char perms[5 ]) +{ +FILE *f; +size_t offset; +int flags, err; + + f =fopen ("/proc/self/maps" ,"r" ); +if (!f) +return -errno; + + err =procmap_query (fileno (f), addr,0 , start, end, &offset, &flags); +#ifdef PROCMAP_QUERY +if (!err && perms) { +snprintf (perms,5 ,"%c%c%c%c" , + (flags &PROCMAP_QUERY_VMA_READABLE ) ?'r' :'-' , + (flags &PROCMAP_QUERY_VMA_WRITABLE ) ?'w' :'-' , + (flags &PROCMAP_QUERY_VMA_EXECUTABLE ) ?'x' :'-' , + (flags &PROCMAP_QUERY_VMA_SHARED ) ?'s' :'p' ); + } +#endif +fclose (f); +return err; +} +ssize_t get_uprobe_offset (const void *addr) {size_t start, base, end;
@@ -352,7 +378,8 @@ ssize_t get_uprobe_offset(const void *addr)return -errno;/* requested executable VMA only */ - err =procmap_query (fileno (f), addr,PROCMAP_QUERY_VMA_EXECUTABLE , &start, &base, &flags); + err =procmap_query (fileno (f), addr,PROCMAP_QUERY_VMA_EXECUTABLE , + &start, &end, &base, &flags);if (err == -EOPNOTSUPP ) {bool found = false;
@@ -418,7 +445,8 @@ ssize_t get_rel_offset(uintptr_t addr)if (!f)return -errno; - err =procmap_query (fileno (f), (const void *)addr,0 , &start, &offset, &flags); + err =procmap_query (fileno (f), (const void *)addr,0 , + &start, &end, &offset, &flags);if (err ==0 ) {fclose (f);return (size_t )addr - start + offset;
tools/testing/selftests/bpf/trace_helpers.h +1 -0
@@ -59,6 +59,7 @@ int read_trace_pipe_iter(void (*cb)(const char *str, void *data),ssize_t get_uprobe_offset (const void *addr);ssize_t get_rel_offset (uintptr_t addr); +int get_procmap_info (const void *addr,size_t *start,size_t *end,char perms[5 ]);int read_build_id (const char *path,char *build_id,size_t size);
Signature · 1 line
2.54.0
Jiri Olsa
reviewerPATCH 4/8 selftests/bpf: Test __srcptr kfunc arguments
In reply to Jiri OlsaVerify that __srcptr kfunc arguments accept read-only global data, writable stack data, and a pointer to a read-only struct of scalars.
Also verify that read-only data is rejected when passed to a kfunc argument that requires writable memory.
--- .../testing/selftests/bpf/prog_tests/kfunc_call.c | 2 ++ .../testing/selftests/bpf/progs/kfunc_call_fail.c | 10 ++++++++++ .../testing/selftests/bpf/progs/kfunc_call_test.c | 14 ++++++++++++++ .../testing/selftests/bpf/test_kmods/bpf_testmod.c | 10 ++++++++++ .../selftests/bpf/test_kmods/bpf_testmod_kfunc.h | 2 ++ 5 files changed, 38 insertions(+)
tools/testing/selftests/bpf/prog_tests/kfunc_call.c +2 -0
@@ -63,6 +63,7 @@ static struct kfunc_test_params kfunc_tests[] = {*/ SYSCALL_NULL_CTX_FAIL (kfunc_syscall_test_fail, -EINVAL ,"processed 4 insns" ),SYSCALL_NULL_CTX_FAIL (kfunc_syscall_test_null_fail, -EINVAL ,"processed 4 insns" ), +TC_FAIL (kfunc_call_test_mem_len_fail_readonly,0 ,"write into map forbidden" ),TC_FAIL (kfunc_call_test_get_mem_fail_rdonly,0 ,"R0 cannot write into rdonly_mem" ),TC_FAIL (kfunc_call_test_get_mem_fail_use_after_free,0 ,"invalid mem access 'scalar'" ), TC_FAIL(kfunc_call_test_get_mem_fail_oob, 0, "min value is outside of the allowed memory range"),
@@ -82,6 +83,7 @@ static struct kfunc_test_params kfunc_tests[] = {TC_TEST (kfunc_call_test5_asm,0 ),TC_TEST (kfunc_call_test_ref_btf_id,0 ),TC_TEST (kfunc_call_test_get_mem,42 ), +TC_TEST (kfunc_call_test_srcptr,0 ),SYSCALL_TEST (kfunc_syscall_test,0 ),SYSCALL_NULL_CTX_TEST (kfunc_syscall_test_null,0 ), TC_TEST(kfunc_call_test_static_unused_arg, 0),
tools/testing/selftests/bpf/progs/kfunc_call_fail.c +10 -0
@@ -4,6 +4,8 @@#include <bpf/bpf_helpers.h> #include "../test_kmods/bpf_testmod_kfunc.h" +static const char mem_len_rodata[] ="read-only data" ; + static struct bpf_spin_lock kfunc_call_lock SEC(".data.A");SEC ("?tc" )
@@ -47,6 +49,14 @@ int kfunc_syscall_test_null_fail(struct syscall_test_args *args)return 0 ; } +SEC("?tc") +int kfunc_call_test_mem_len_fail_readonly (struct __sk_buff *skb) +{ +bpf_kfunc_call_test_mem_len_pass1 ((void *)mem_len_rodata, +sizeof (mem_len_rodata)); +return 0 ; +} + SEC("?tc")int kfunc_call_test_get_mem_fail_rdonly (struct __sk_buff *skb) {
tools/testing/selftests/bpf/progs/kfunc_call_test.c +14 -0
@@ -5,6 +5,9 @@#include "bpf_misc.h" #include "../test_kmods/bpf_testmod_kfunc.h" +static const char srcptr_rodata[] ="read-only source" ; +static const struct prog_test_pass1 srcptr_struct_rodata = { .x0 =1 }; + static struct bpf_spin_lock kfunc_call_lock SEC(".data.A");SEC ("tc" )
@@ -208,6 +211,17 @@ int kfunc_call_test_pass(struct __sk_buff *skb)return 0 ; } +SEC("tc") +int kfunc_call_test_srcptr (struct __sk_buff *skb) +{ +char stack_data[] ="read-write source" ; + +bpf_kfunc_call_test_srcptr (srcptr_rodata,sizeof (srcptr_rodata)); +bpf_kfunc_call_test_srcptr (stack_data,sizeof (stack_data)); +bpf_kfunc_call_test_srcptr_struct (&srcptr_struct_rodata); +return 0 ; +} +struct syscall_test_args {__u8 data[16 ];size_t size;
tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +10 -0
@@ -1024,6 +1024,8 @@ BTF_ID_FLAGS(func, bpf_kfunc_arena_args5_test) BTF_ID_FLAGS(func, bpf_kfunc_arena_stack_arg_test) BTF_ID_FLAGS(func, bpf_kfunc_arena_mixed_test) BTF_ID_FLAGS(func, bpf_kfunc_call_test_mem_len_pass1) +BTF_ID_FLAGS(func, bpf_kfunc_call_test_srcptr) +BTF_ID_FLAGS(func, bpf_kfunc_call_test_srcptr_struct) BTF_ID_FLAGS(func, bpf_kfunc_dynptr_test) BTF_ID_FLAGS(func, bpf_kfunc_nested_acquire_nonzero_offset_test, KF_ACQUIRE)BTF_ID_FLAGS (func, bpf_kfunc_nested_acquire_zero_offset_test,KF_ACQUIRE )
@@ -1527,6 +1529,14 @@ __bpf_kfunc void bpf_kfunc_call_test_mem_len_pass1(void *mem, int mem__sz) { } +__bpf_kfunc void bpf_kfunc_call_test_srcptr(const void *src__srcptr, int src__sz) +{ +} + +__bpf_kfunc void bpf_kfunc_call_test_srcptr_struct(const struct prog_test_pass1 *p__srcptr) +{ +} + __bpf_kfunc void bpf_kfunc_call_test_mem_len_fail1(void *mem, int len) { }
tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h +2 -0
@@ -198,6 +198,8 @@ void bpf_kfunc_call_test_release(struct prog_test_ref_kfunc *p) __ksym; void bpf_kfunc_call_test_ref(struct prog_test_ref_kfunc *p) __ksym; void bpf_kfunc_call_test_mem_len_pass1(void *mem, int len) __ksym; +void bpf_kfunc_call_test_srcptr(const void *src__srcptr, int src__sz) __ksym; +void bpf_kfunc_call_test_srcptr_struct(const struct prog_test_pass1 *p__srcptr) __ksym; __u64 bpf_kfunc_arena_arg_test(__u64 *val__arena) __ksym; __u64 bpf_kfunc_arena_cap_test(__u64 *val__arena) __ksym; __u64 bpf_kfunc_arena_cap_nullable_test(__u64 *val__arena__nullable) __ksym;
Signature · 1 line
2.54.0
Jiri Olsa
reviewerPATCH 5/8 selftests/bpf: Test user memory mmap kfuncs
In reply to Jiri OlsaAdd BPF selftests for bpf_user_mmap and bpf_user_munmap from a sleepable uprobe. Check that created mappings have the requested permissions and that unmapped addresses are no longer mapped.
--- Documentation/bpf/kfuncs.rst | 17 +++ .../selftests/bpf/prog_tests/user_mem.c | 112 ++++++++++++++++++ tools/testing/selftests/bpf/progs/user_mem.c | 43 +++++++ 3 files changed, 172 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/user_mem.c create mode 100644 tools/testing/selftests/bpf/progs/user_mem.c
Documentation/bpf/kfuncs.rst +17 -0
@@ -989,3 +989,20 @@ the verifier. bpf_cgroup_ancestor() can be used as follows: BPF provides a set of kfuncs that can be used to query, allocate, mutate, and destroy struct cpumask * objects. Please refer to :ref:`cpumasks-header-label` for more details. + +4.4 User memory kfuncs +---------------------- + +The user memory kfuncs operate on the current process's address space. +``bpf_copy_to_user()`` writes to this address space, including private mappings +that are not writable. A partial copy is not rolled back if it faults. + +``bpf_user_mmap()`` and ``bpf_user_munmap()`` create and remove mappings by +calling the kernel's mapping functions directly rather than issuing +``mmap(2)`` or ``munmap(2)`` syscalls. As a result, seccomp filters do not +mediate these mapping operations. A seccomp filter can still allow or deny the +syscall that triggers a BPF program, but it cannot separately filter the +mapping performed by these kfuncs. + +.. kernel-doc:: kernel/bpf/helpers.c + :identifiers: bpf_copy_to_user bpf_user_mmap bpf_user_munmap
tools/testing/selftests/bpf/prog_tests/user_mem.c new file +112 -0
112 lines not on this page · raw message
tools/testing/selftests/bpf/progs/user_mem.c new file +43 -0
43 lines not on this page · raw message
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH bpf-next 5/8] selftests/bpf: Test user memory mmap kfuncs
TL;DR: Nits only: the changelog omits the kfuncs.rst documentation added by the patch, which may belong in another patch; also an unneeded <sys/personality.h> include and a never-used NULL perms…
Quoted 1 line
selftests/bpf: Test user memory mmap kfuncs
TL;DR: Nits only: the changelog omits the kfuncs.rst documentation added by the patch, which may belong in another patch; also an unneeded <sys/personality.h> include and a never-used NULL perms path in check_user_mem_vma().
Quoted 8 lines
selftests/bpf: Test user memory mmap kfuncs
Add BPF selftests for
bpf_user_mmapandbpf_user_munmapfrom a sleepable uprobe. Check that created mappings have the requested permissions and that unmapped addresses are no longer mapped.Assisted-by Codex:GPT-6Signed-off-by Jiri Olsa <[email protected]>
This isn't a bug, but the changelog only describes the new selftests, while the patch also adds a "4.4 User memory kfuncs" section to Documentation/bpf/kfuncs.rst.
That section documents bpf_copy_to_user() as well, which is not exercised by the tests added here, and it states that seccomp filters do not mediate the mappings made by these kfuncs.
Could the kfuncs.rst section go into "bpf: Add user memory kfuncs" or a separate documentation patch, or at least be mentioned in this changelog?
tools/testing/selftests/bpf/prog_tests/user_mem.c line 5
Quoted 11 lines
tools/testing/selftests/bpf/prog_tests/user_mem.c new file +5
-0@@ -0,0 +1,112 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include <test_progs.h> +#include <sys/mman.h> +#include <sys/personality.h>
^^^^^^^^^^^^^^^^
This isn't a bug, but is <sys/personality.h> needed here? Nothing in this file calls personality() or uses a PER_*, READ_IMPLIES_EXEC or ADDR_* constant.
Quoted 1 line
tools/testing/selftests/bpf/prog_tests/user_mem.c new file +1
-0+#include <unistd.h>
[ ... ]
Quoted 2 lines
tools/testing/selftests/bpf/prog_tests/user_mem.c new file +2
-0+static bool check_user_mem_vma (void *addr,unsigned long size,const char *perms) +{
[ ... ]
tools/testing/selftests/bpf/prog_tests/user_mem.c line 35
Quoted 1 line
tools/testing/selftests/bpf/prog_tests/user_mem.c new file +1
-0+return !perms ||ASSERT_STREQ (vma_perms, perms,"vma_perms" );
^^^^^^^^
This isn't a bug, but does check_user_mem_vma() need to handle a NULL perms? test_mapping() passes the literal "rw-p", and the later patches in the series pass "rw-p" and "r-xp", so the "!perms ||" branch is never taken.
Quoted 1 line
+}
[ ... ]
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/40959191771
Re: [PATCH bpf-next 5/8] selftests/bpf: Test user memory mmap kfuncs
Quoted 20 lines · On Thu, Oct 29, 2026 at 09:59:31PM +0000, [email protected] wrote:
Quoted 1 line
selftests/bpf: Test user memory mmap kfuncs
TL;DR: Nits only: the changelog omits the
kfuncs.rstdocumentation added by the patch, which may belong in another patch; also an unneeded <sys/personality.h> include and a never-used NULL perms path incheck_user_mem_vma().Quoted 8 lines
selftests/bpf: Test user memory mmap kfuncs
Add BPF selftests for
bpf_user_mmapandbpf_user_munmapfrom a sleepable uprobe. Check that created mappings have the requested permissions and that unmapped addresses are no longer mapped.Assisted-by Codex:GPT-6Signed-off-by Jiri Olsa <[email protected]>This isn't a bug, but the changelog only describes the new selftests, while the patch also adds a "4.4 User memory kfuncs" section to
Documentation/bpf/kfuncs.rst.
Quoted 6 lines
That section documents
bpf_copy_to_user()as well, which is not exercised by the tests added here, and it states that seccomp filters do not mediate the mappings made by these kfuncs.Could the
kfuncs.rstsection go into "bpf: Add user memory kfuncs" or a separate documentation patch, or at least be mentioned in this changelog?
yes, it needs to go there, will fix
jirka
Jiri Olsa
reviewerPATCH 6/8 selftests/bpf: Test user memory page exec
In reply to Jiri OlsaUse bpf_copy_to_user from a uprobe to write architecture-specific instructions into user memory, then execute them on x86-64 and arm64.
Test execution from a mapping created with read and execute permissions, and from individual pages made executable with mprotect.
--- .../selftests/bpf/prog_tests/user_mem.c | 75 +++++++++++++++++++ tools/testing/selftests/bpf/progs/user_mem.c | 22 ++++++ 2 files changed, 97 insertions(+)
tools/testing/selftests/bpf/prog_tests/user_mem.c +75 -0
@@ -11,6 +11,7 @@enum {USER_MEM_MMAP =1 ,USER_MEM_UNMAP , +USER_MEM_COPY_INSNS , }; static noinline void user_mem_trigger(void)
@@ -92,6 +93,76 @@ static void test_mapping(struct user_mem *skel)munmap ((void *)addr, len); } +static bool user_mem_copy_insns (struct user_mem *skel,unsigned long addr) +{ + skel->bss->user_addr = addr; + skel->bss->action =USER_MEM_COPY_INSNS ; +user_mem_trigger (); + +return ASSERT_OK (skel->bss->ret,"user_write" ); +} + +static void test_execute_user_page (struct user_mem *skel) +{ +long page_size =sysconf (_SC_PAGESIZE); +unsigned long addr = ~0UL ; + +#if !defined(__x86_64__) && !defined(__aarch64__) +test__skip (); +return ; +#endif + +if (!user_mem_mmap (skel, page_size,PROT_READ |PROT_EXEC , &addr)) +return ; +if (!check_user_mem_vma ((void *)addr, page_size,"r-xp" )) + goto out; +if (!user_mem_copy_insns (skel, addr)) + goto out; + +/* Execute user page. */ +ASSERT_EQ (((int (*)(void ))addr)(),42 ,"execute_user_page" ); + +out: +if (addr != ~0UL ) +munmap ((void *)addr, page_size); +} + +static void test_partial_exec (struct user_mem *skel) +{ +long page_size =sysconf (_SC_PAGESIZE); +unsigned long addr = ~0UL ; + +#if !defined(__x86_64__) && !defined(__aarch64__) +test__skip (); +return ; +#endif + +if (!user_mem_mmap (skel, page_size *3 ,PROT_READ |PROT_WRITE , &addr)) +return ; + +/* Copy instructions to middle page. */ +if (!user_mem_copy_insns (skel, addr + page_size)) + goto out; + +/* Set +x to middle page. */ +if (!ASSERT_OK (mprotect ((void *)(addr + page_size), page_size, +PROT_READ |PROT_EXEC ),"middle_page_mprotect" )) + goto out; + +if (!check_user_mem_vma ((void *)addr, page_size,"rw-p" ) || + !check_user_mem_vma ((void *)(addr + page_size), page_size,"r-xp" ) || + !check_user_mem_vma ((void *)(addr +2 * page_size), page_size,"rw-p" )) + goto out; + +/* Execute middle page. */ +if (!ASSERT_EQ (((int (*)(void ))(addr + page_size))(),42 ,"execute_middle_page" )) + goto out; + +out: +if (addr != ~0UL ) +munmap ((void *)addr, page_size *3 ); +} +void test_user_mem (void ) {struct user_mem *skel;
@@ -106,6 +177,10 @@ void test_user_mem(void)if (test__start_subtest ("mapping" ))test_mapping (skel); +if (test__start_subtest ("execute_user_page" )) +test_execute_user_page (skel); +if (test__start_subtest ("partial_exec" )) +test_partial_exec (skel); out:user_mem__destroy (skel);
tools/testing/selftests/bpf/progs/user_mem.c +22 -0
@@ -10,10 +10,13 @@ char _license[] SEC("license") = "GPL"; extern unsigned long bpf_user_mmap(unsigned long len, unsigned long prot) __ksym; extern int bpf_user_munmap(unsigned long addr, unsigned long len) __ksym; +extern int bpf_copy_to_user (void *unsafe_ptr__ign,const void *src__srcptr, + u32 src__sz, u64 flags) __ksym; enum {USER_MEM_MMAP =1 ,USER_MEM_UNMAP , + USER_MEM_COPY_INSNS, };int monitored_pid;
@@ -24,6 +27,22 @@ unsigned long user_addr;unsigned long map_len;unsigned long map_prot; +#if defined(__TARGET_ARCH_x86) +static const unsigned char insns[] = { +0xf3 ,0x0f ,0x1e ,0xfa ,/* endbr64 */ +0xb8 ,42 ,0 ,0 ,0 ,0xc3 ,/* mov $42, %eax; ret */ +}; +#elif defined(__TARGET_ARCH_arm64) +static const unsigned char insns[] = { +0x5f ,0x24 ,0x03 ,0xd5 ,/* bti c */ +0x40 ,0x05 ,0x80 ,0x52 ,/* mov w0, #42 */ +0xc0 ,0x03 ,0x5f ,0xd6 ,/* ret */ +}; +#else +/* The execute tests are skipped on other architectures. */ +static const unsigned char insns[] = {0 }; +#endif + SEC("uprobe.s//proc/self/exe:user_mem_trigger")int user_mem_test (struct pt_regs *ctx) {
@@ -37,6 +56,9 @@ int user_mem_test(struct pt_regs *ctx)case USER_MEM_UNMAP : ret =bpf_user_munmap (user_addr, map_len);break ; +case USER_MEM_COPY_INSNS : + ret =bpf_copy_to_user ((void *)user_addr, insns,sizeof (insns),0 ); +break ; }return 0 ;
Signature · 1 line
2.54.0
Jiri Olsa
reviewerPATCH 7/8 selftests/bpf: Test bpf_copy_to_user kfunc
In reply to Jiri OlsaTest that bpf_copy_to_user can fault in a paged-out page and write to a read-only mapping. Also verify that a partial copy crossing into an unmapped page returns -EFAULT while leaving the copied byte in place.
--- .../selftests/bpf/prog_tests/user_mem.c | 106 ++++++++++++++++++ tools/testing/selftests/bpf/progs/user_mem.c | 17 +++ 2 files changed, 123 insertions(+)
tools/testing/selftests/bpf/prog_tests/user_mem.c +106 -0
@@ -12,8 +12,11 @@ enum {USER_MEM_MMAP =1 ,USER_MEM_UNMAP ,USER_MEM_COPY_INSNS , + USER_MEM_COPY, }; +static const char data[] ="fault-enabled BPF write" ; + static noinline void user_mem_trigger(void) { asmvolatile ("" :::"memory" );
@@ -102,6 +105,18 @@ static bool user_mem_copy_insns(struct user_mem *skel, unsigned long addr)return ASSERT_OK (skel->bss->ret,"user_write" ); } +static int user_mem_copy (struct user_mem *skel,unsigned long addr, +u32 size,u64 flags) +{ + skel->bss->user_addr = addr; + skel->bss->copy_size = size; + skel->bss->copy_flags = flags; + skel->bss->action =USER_MEM_COPY ; +user_mem_trigger (); + +return skel->bss->ret; +} +static void test_execute_user_page (struct user_mem *skel) {long page_size =sysconf (_SC_PAGESIZE);
@@ -163,6 +178,91 @@ static void test_partial_exec(struct user_mem *skel)munmap ((void *)addr, page_size *3 ); } +static void test_faultable_write (struct user_mem *skel) +{ +long page_size =sysconf (_SC_PAGESIZE); +unsigned long addr = ~0UL ; +unsigned char vec; +int i; + +if (!user_mem_mmap (skel, page_size,PROT_READ |PROT_WRITE , &addr)) +return ; + +/* Make sure there is a private resident page to page out. */ +memset ((void *)addr,0 , page_size); + +/* Retry pageout until mincore confirms that the page is no longer resident. */ +for (i =0 ; i <500 ; i++) { +if (madvise ((void *)addr, page_size,MADV_PAGEOUT )) { +test__skip (); + goto out; + } +if (!ASSERT_OK (mincore ((void *)addr, page_size, &vec),"mincore" )) + goto out; +if (!(vec &1 )) +break ; +usleep (100 ); + } +if (i ==500 ) { +test__skip (); + goto out; + } + +/* ..and do the copy to it. */ +if (!ASSERT_OK (user_mem_copy (skel, addr,sizeof (data),0 ),"user_write_data" )) + goto out; + +ASSERT_EQ (memcmp ((void *)addr, data,sizeof (data)),0 ,"written_data" ); + +out: +if (addr != ~0UL ) +munmap ((void *)addr, page_size); +} + +static void test_copy_to_readonly_mapping (struct user_mem *skel) +{ +long page_size =sysconf (_SC_PAGESIZE); +unsigned long addr = ~0UL ; + +if (!user_mem_mmap (skel, page_size,PROT_READ , &addr)) +return ; + +/* Assert we can copy to read-only mapping. */ +if (!ASSERT_OK (user_mem_copy (skel, addr,sizeof (data),0 ),"readonly_copy" )) + goto out; + +ASSERT_EQ (memcmp ((void *)addr, data,sizeof (data)),0 ,"readonly_copy_data" ); + +out: +if (addr != ~0UL ) +munmap ((void *)addr, page_size); +} + +static void test_partial_copy_fault (struct user_mem *skel) +{ +long page_size =sysconf (_SC_PAGESIZE); +bool second_page_unmapped = false; +unsigned long addr = ~0UL ; + +if (!user_mem_mmap (skel, page_size *2 ,PROT_READ |PROT_WRITE , &addr)) +return ; + +if (!user_mem_munmap (skel, addr + page_size, page_size)) + goto out; + + second_page_unmapped = true; + +ASSERT_EQ (user_mem_copy (skel, addr + page_size -1 ,sizeof (data),0 ), + -EFAULT ,"partial_copy" ); + +/* The byte before the fault stays written. */ +ASSERT_EQ (((char *)addr)[page_size -1 ],'f' ,"partial_copy_data" ); + +out: +if (addr != ~0UL ) +munmap ((void *)addr, second_page_unmapped ? page_size : page_size *2 ); +} +void test_user_mem (void ) {struct user_mem *skel;
@@ -181,6 +281,12 @@ void test_user_mem(void)test_execute_user_page (skel);if (test__start_subtest ("partial_exec" ))test_partial_exec (skel); +if (test__start_subtest ("faultable_write" )) +test_faultable_write (skel); +if (test__start_subtest ("copy_to_readonly_mapping" )) +test_copy_to_readonly_mapping (skel); +if (test__start_subtest ("partial_copy_fault" )) +test_partial_copy_fault (skel); out:user_mem__destroy (skel);
tools/testing/selftests/bpf/progs/user_mem.c +17 -0
@@ -17,6 +17,7 @@ enum {USER_MEM_MMAP =1 ,USER_MEM_UNMAP ,USER_MEM_COPY_INSNS , + USER_MEM_COPY, };int monitored_pid;
@@ -27,6 +28,9 @@ unsigned long user_addr;unsigned long map_len;unsigned long map_prot; +u64 copy_flags; +u32 copy_size; +#if defined(__TARGET_ARCH_x86)static const unsigned char insns[] = { 0xf3, 0x0f, 0x1e, 0xfa, /* endbr64 */
@@ -43,9 +47,13 @@ static const unsigned char insns[] = {static const unsigned char insns[] = {0 };#endif +static const char data[] ="fault-enabled BPF write" ; + SEC("uprobe.s//proc/self/exe:user_mem_trigger")int user_mem_test (struct pt_regs *ctx) { +u32 size; +if ((bpf_get_current_pid_tgid () >>32 ) != monitored_pid)return 0 ;
@@ -59,6 +67,15 @@ int user_mem_test(struct pt_regs *ctx)case USER_MEM_COPY_INSNS : ret =bpf_copy_to_user ((void *)user_addr, insns,sizeof (insns),0 );break ; +case USER_MEM_COPY : +/* Bound the kfunc length using a local copy of copy_size. */ + size = copy_size; +if (size >sizeof (data)) { + ret =-22 ;/* -EINVAL */ +break ; + } + ret =bpf_copy_to_user ((void *)user_addr, data, size, copy_flags); +break ; }return 0 ;
Signature · 1 line
2.54.0
Jiri Olsa
reviewerPATCH 8/8 selftests/bpf: Test user memory kfunc errors
In reply to Jiri OlsaTest invalid copy arguments, zero-length mapping operations, partial copy faults, and copying to read-only mappings. Verify that a failed unmap leaves the mapping intact, and that non-sleepable programs and programs without CAP_SYS_ADMIN cannot call bpf_user_mmap.
--- .../selftests/bpf/prog_tests/user_mem.c | 45 +++++++++++++++++++ tools/testing/selftests/bpf/progs/user_mem.c | 29 ++++++++++++ 2 files changed, 74 insertions(+)
tools/testing/selftests/bpf/prog_tests/user_mem.c +45 -0
@@ -263,10 +263,49 @@ static void test_partial_copy_fault(struct user_mem *skel)munmap ((void *)addr, second_page_unmapped ? page_size : page_size *2 ); } +static void test_zero_length_mapping (struct user_mem *skel) +{ + skel->bss->map_len =0 ; + skel->bss->map_prot =PROT_READ |PROT_WRITE ; + skel->bss->action =USER_MEM_MMAP ; +user_mem_trigger (); + +ASSERT_EQ (skel->bss->user_addr, (unsigned long )-EINVAL ,"zero_length_mapping" ); +} + +static void test_failed_unmap (struct user_mem *skel) +{ +long page_size =sysconf (_SC_PAGESIZE); +unsigned long addr = ~0UL ; + +if (!user_mem_mmap (skel, page_size,PROT_READ |PROT_WRITE , &addr)) +return ; + + skel->bss->user_addr = addr; + skel->bss->map_len =0 ; + skel->bss->action =USER_MEM_UNMAP ; +user_mem_trigger (); + +if (ASSERT_EQ (skel->bss->ret, -EINVAL ,"zero_length_unmap" )) +check_user_mem_vma ((void *)addr, page_size,"rw-p" ); + +munmap ((void *)addr, page_size); +} + +static void test_invalid_copy (struct user_mem *skel) +{ +ASSERT_EQ (user_mem_copy (skel,1 ,sizeof (data),0 ), -EFAULT ,"bad_copy" ); +ASSERT_EQ (user_mem_copy (skel,1 ,sizeof (data),1 ), -EINVAL ,"bad_copy_flags" ); +ASSERT_EQ (user_mem_copy (skel,1 ,0 ,0 ),0 ,"zero_length_copy" ); +ASSERT_EQ (user_mem_copy (skel,1 ,0 ,1 ), -EINVAL ,"zero_length_copy_bad_flags" ); +} +void test_user_mem (void ) {struct user_mem *skel; +RUN_TESTS (user_mem); + skel =user_mem__open_and_load ();if (!ASSERT_OK_PTR (skel,"open_and_load" ))return ;
@@ -287,6 +326,12 @@ void test_user_mem(void)test_copy_to_readonly_mapping (skel);if (test__start_subtest ("partial_copy_fault" ))test_partial_copy_fault (skel); +if (test__start_subtest ("failed_unmap" )) +test_failed_unmap (skel); +if (test__start_subtest ("invalid_copy" )) +test_invalid_copy (skel); +if (test__start_subtest ("zero_length_mapping" )) +test_zero_length_mapping (skel); out:user_mem__destroy (skel);
tools/testing/selftests/bpf/progs/user_mem.c +29 -0
@@ -80,3 +80,32 @@ int user_mem_test(struct pt_regs *ctx)return 0 ; } + +SEC("?uprobe//proc/self/exe:user_mem_trigger") +__failure +__msg("program must be sleepable to call sleepable kfunc bpf_user_mmap") +__msg("cannot be used in non-sleepable program") +int user_mem_nonsleepable (struct pt_regs *ctx) +{ +bpf_user_mmap (4096 ,1 ); +return 0 ; +} + +SEC("?kprobe") +__failure +__msg("program must be sleepable to call sleepable kfunc bpf_user_mmap") +__msg("cannot be used in non-sleepable program") +int user_mem_kprobe (struct pt_regs *ctx) +{ +bpf_user_mmap (4096 ,1 ); +return 0 ; +} + +SEC("?uprobe.s//proc/self/exe:user_mem_trigger") +__failure_unpriv __caps_unpriv(CAP_BPF | CAP_PERFMON) +__msg_unpriv("calling kernel function bpf_user_mmap is not allowed") +int user_mem_no_sysadmin (struct pt_regs *ctx) +{ +bpf_user_mmap (4096 ,1 ); +return 0 ; +}
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH bpf-next 8/8] selftests/bpf: Test user memory kfunc errors
TL;DR: Commit message only: it claims tests for partial copy faults and read-only mappings that were added by the previous patch, and omits the new RUN_TESTS(user_mem) verifier-failure tests.
Quoted 1 line
selftests/bpf: Test user memory kfunc errors
TL;DR: Commit message only: it claims tests for partial copy faults and read-only mappings that were added by the previous patch, and omits the new RUN_TESTS(user_mem) verifier-failure tests.
Quoted 9 lines
selftests/bpf: Test user memory kfunc errors
Test invalid copy arguments, zero-length mapping operations, partial copy faults, and copying to read-only mappings. Verify that a failed unmap leaves the mapping intact, and that non-sleepable programs and programs without
CAP_SYS_ADMINcannot callbpf_user_mmap.Assisted-by Codex:GPT-6Signed-off-by Jiri Olsa <[email protected]>
The changelog says this commit tests "partial copy faults, and copying to read-only mappings". Is that accurate for this patch?
test_partial_copy_fault() and test_copy_to_readonly_mapping() were added by the previous commit 8f0bac0f80cd ("selftests/bpf: Test bpf_copy_to_user kfunc"), whose changelog already describes them, and this diff does not touch them.
What this diff adds is test_zero_length_mapping(), test_failed_unmap(), test_invalid_copy(), the three verifier-failure programs (user_mem_nonsleepable, user_mem_kprobe and user_mem_no_sysadmin), and the RUN_TESTS(user_mem) call that runs them.
The changelog also does not mention RUN_TESTS(user_mem), which turns the existing user_mem_test program into a privileged load-only subtest.
Could the changelog be reworded to describe what this patch adds?
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/40959191771
Re: [PATCH bpf-next 8/8] selftests/bpf: Test user memory kfunc errors
Quoted 29 lines · On Thu, Oct 29, 2026 at 09:59:33PM +0000, [email protected] wrote:
Quoted 1 line
selftests/bpf: Test user memory kfunc errors
TL;DR: Commit message only: it claims tests for partial copy faults and read-only mappings that were added by the previous patch, and omits the new
RUN_TESTS(user_mem)verifier-failure tests.Quoted 9 lines
selftests/bpf: Test user memory kfunc errors
Test invalid copy arguments, zero-length mapping operations, partial copy faults, and copying to read-only mappings. Verify that a failed unmap leaves the mapping intact, and that non-sleepable programs and programs without
CAP_SYS_ADMINcannot callbpf_user_mmap.Assisted-by Codex:GPT-6Signed-off-by Jiri Olsa <[email protected]>The changelog says this commit tests "partial copy faults, and copying to read-only mappings". Is that accurate for this patch?
test_partial_copy_fault()andtest_copy_to_readonly_mapping()were added by the previous commit8f0bac0f80cd("selftests/bpf: Testbpf_copy_to_userkfunc"), whose changelog already describes them, and this diff does not touch them.What this diff adds is
test_zero_length_mapping(),test_failed_unmap(),test_invalid_copy(), the three verifier-failure programs (user_mem_nonsleepable,user_mem_kprobeanduser_mem_no_sysadmin), and theRUN_TESTS(user_mem)call that runs them.
Quoted 4 lines
The changelog also does not mention
RUN_TESTS(user_mem), which turns the existinguser_mem_testprogram into a privileged load-only subtest.Could the changelog be reworded to describe what this patch adds?
yep, will fix
jirka