Dynptr refactorings
Subsystems (4)
Kumar Kartikeya Dwivedi
maintainerv2 cover Dynptr refactorings
This is part 1 of https://lore.kernel.org/bpf/[email protected].
This thread also gives some background on why the refactor is being done:
https://lore.kernel.org/bpf/CAEf4Bzb4beTHgVo+G+jehSj8oCeAjRbRcm6MRe=Gr+cajRBwEw@mail.gmail.com
As requested in patch 6 by Alexei, it only includes patches which refactors the code, on top of which further fixes will be made in part
2. The refactor itself fixes another issue as a side effect. No functional change is intended (except a few modified log messages).
Changelog:
----------
v1 -> v2
v1: https://lore.kernel.org/bpf/[email protected]
* Address feedback from Joanne and David, add acks
Fixes v1 -> v1
Fixes v1: https://lore.kernel.org/bpf/[email protected]
* Collect acks from Joanne and David
* Fix misc nits pointed out by Joanne, David
* Split move of reg->off alignment check for dynptr into separate change (Alexei)
Kumar Kartikeya Dwivedi (7):
bpf: Refactor ARG_PTR_TO_DYNPTR checks into process_dynptr_func
bpf: Propagate errors from process_* checks in check_func_arg
bpf: Rework process_dynptr_func
bpf: Rework check_func_arg_reg_off
bpf: Move PTR_TO_STACK alignment check to process_dynptr_func
bpf: Use memmove for bpf_dynptr_{read,write} selftests/bpf: Add test for dynptr reinit in user_ringbuf callback
include/linux/bpf.h | 4 +- include/linux/bpf_verifier.h | 8 +- include/uapi/linux/bpf.h | 8 +- kernel/bpf/helpers.c | 30 +- kernel/bpf/verifier.c | 435 ++++++++++++------ scripts/bpf_doc.py | 1 + tools/include/uapi/linux/bpf.h | 8 +- .../bpf/prog_tests/kfunc_dynptr_param.c | 7 +- .../selftests/bpf/prog_tests/user_ringbuf.c | 6 +- .../bpf/progs/test_kfunc_dynptr_param.c | 12 - .../selftests/bpf/progs/user_ringbuf_fail.c | 51 +- tools/testing/selftests/bpf/verifier/calls.c | 2 +- .../testing/selftests/bpf/verifier/ringbuf.c | 2 +- 13 files changed, 371 insertions(+), 203 deletions(-)
base-commit: e9b4aeed56699b469206d05e706ddf2db95700a9
Signature · 1 line
2.38.1
Kumar Kartikeya Dwivedi
maintainerPATCH v2 1/7 bpf: Refactor ARG_PTR_TO_DYNPTR checks into process_dynptr_func
ARG_PTR_TO_DYNPTR is akin to ARG_PTR_TO_TIMER, ARG_PTR_TO_KPTR, where the underlying register type is subjected to more special checks to determine the type of object represented by the pointer and its state consistency.
Move dynptr checks to their own 'process_dynptr_func' function so that is consistent and in-line with existing code. This also makes it easier to reuse this code for kfunc handling.
Then, reuse this consolidated function in kfunc dynptr handling too. Note that for kfuncs, the arg_type constraint of DYNPTR_TYPE_LOCAL has been lifted.
--- include/linux/bpf_verifier.h | 8 +- kernel/bpf/verifier.c | 134 +++++++++--------- .../bpf/prog_tests/kfunc_dynptr_param.c | 7 +- .../bpf/progs/test_kfunc_dynptr_param.c | 12 -- 4 files changed, 75 insertions(+), 86 deletions(-)
include/linux/bpf_verifier.h +3 -5
@@ -615,11 +615,9 @@ int check_func_arg_reg_off(struct bpf_verifier_env *env, enum bpf_arg_type arg_type);int check_mem_reg (struct bpf_verifier_env *env,struct bpf_reg_state *reg,u32 regno,u32 mem_size); -bool is_dynptr_reg_valid_init (struct bpf_verifier_env *env, -struct bpf_reg_state *reg); -bool is_dynptr_type_expected (struct bpf_verifier_env *env, -struct bpf_reg_state *reg, -enum bpf_arg_type arg_type); +struct bpf_call_arg_meta ; +int process_dynptr_func (struct bpf_verifier_env *env,int regno, +enum bpf_arg_type arg_type,struct bpf_call_arg_meta *meta);/* this lives here instead of in bpf.h because it needs to dereference tgt_prog */ static inline u64 bpf_trampoline_compute_key(const struct bpf_prog *tgt_prog,
kernel/bpf/verifier.c +70 -64
167 lines not on this page · raw message
tools/testing/selftests/bpf/prog_tests/kfunc_dynptr_param.c +2 -5
@@ -18,11 +18,8 @@ static struct {const char *expected_verifier_err_msg;int expected_runtime_err; } kfunc_dynptr_tests[] = { - {"dynptr_type_not_supp", - "arg#0 pointer type STRUCT bpf_dynptr_kern points to unsupported dynamic pointer type", 0}, - {"not_valid_dynptr", - "arg#0 pointer type STRUCT bpf_dynptr_kern must be valid and initialized", 0}, - {"not_ptr_to_stack", "arg#0 expected pointer to stack", 0}, + {"not_valid_dynptr", "Expected an initialized dynptr as arg #1", 0}, + {"not_ptr_to_stack", "arg#0 expected pointer to stack or dynptr_ptr", 0}, {"dynptr_data_null", NULL, -EBADMSG}, };
tools/testing/selftests/bpf/progs/test_kfunc_dynptr_param.c +0 -12
@@ -32,18 +32,6 @@ int err, pid; char _license[] SEC("license") = "GPL"; -SEC("?lsm.s/bpf") -int BPF_PROG (dynptr_type_not_supp ,int cmd,union bpf_attr *attr, -unsigned int size) -{ -char write_data[64 ] ="hello there, world!!" ; -struct bpf_dynptr ptr; - -bpf_ringbuf_reserve_dynptr (&ringbuf,sizeof (write_data),0 , &ptr); - -return bpf_verify_pkcs7_signature (&ptr, &ptr,NULL ); -} - SEC("?lsm.s/bpf")int BPF_PROG (not_valid_dynptr ,int cmd,union bpf_attr *attr,unsigned int size) {
Signature · 1 line
2.38.1
Kumar Kartikeya Dwivedi
maintainerPATCH v2 2/7 bpf: Propagate errors from process_* checks in check_func_arg
In reply to Kumar Kartikeya DwivediCurrently, we simply ignore the errors in process_spin_lock, process_timer_func, process_kptr_func, process_dynptr_func. Instead, bubble up the error by storing and checking err variable.
--- kernel/bpf/verifier.c | 25 +++++++++++++++---------- 1 file changed, 15 insertions(+), 10 deletions(-)
kernel/bpf/verifier.c +15 -10
@@ -6412,19 +6412,22 @@ static int check_func_arg(struct bpf_verifier_env *env, u32 arg,break ;case ARG_PTR_TO_SPIN_LOCK :if (meta->func_id == BPF_FUNC_spin_lock) { -if (process_spin_lock (env, regno, true)) -return -EACCES ; + err =process_spin_lock (env, regno, true); +if (err) +return err; }else if (meta->func_id == BPF_FUNC_spin_unlock) { -if (process_spin_lock (env, regno, false)) -return -EACCES ; + err =process_spin_lock (env, regno, false); +if (err) +return err; }else {verbose (env,"verifier internal error\n" );return -EFAULT ; }break ;case ARG_PTR_TO_TIMER : -if (process_timer_func (env, regno, meta)) -return -EACCES ; + err =process_timer_func (env, regno, meta); +if (err) +return err;break ;case ARG_PTR_TO_FUNC : meta->subprogno = reg->subprogno;
@@ -6447,8 +6450,9 @@ static int check_func_arg(struct bpf_verifier_env *env, u32 arg, err =check_mem_size_reg (env, reg, regno, true, meta);break ;case ARG_PTR_TO_DYNPTR : -if (process_dynptr_func (env, regno, arg_type, meta)) -return -EACCES ; + err =process_dynptr_func (env, regno, arg_type, meta); +if (err) +return err;break ;case ARG_CONST_ALLOC_SIZE_OR_ZERO :if (!tnum_is_const (reg->var_off)) {
@@ -6515,8 +6519,9 @@ static int check_func_arg(struct bpf_verifier_env *env, u32 arg,break ; }case ARG_PTR_TO_KPTR : -if (process_kptr_func (env, regno, meta)) -return -EACCES ; + err =process_kptr_func (env, regno, meta); +if (err) +return err;break ; }
Signature · 1 line
2.38.1
Kumar Kartikeya Dwivedi
maintainerPATCH v2 3/7 bpf: Rework process_dynptr_func
In reply to Kumar Kartikeya DwivediRecently, user ringbuf support introduced a PTR_TO_DYNPTR register type for use in callback state, because in case of user ringbuf helpers, there is no dynptr on the stack that is passed into the callback. To reflect such a state, a special register type was created.
However, some checks have been bypassed incorrectly during the addition of this feature. First, for arg_type with MEM_UNINIT flag which initialize a dynptr, they must be rejected for such register type. Secondly, in the future, there are plans to add dynptr helpers that operate on the dynptr itself and may change its offset and other properties.
In all of these cases, PTR_TO_DYNPTR shouldn't be allowed to be passed to such helpers, however the current code simply returns 0.
The rejection for helpers that release the dynptr is already handled.
For fixing this, we take a step back and rework existing code in a way that will allow fitting in all classes of helpers and have a coherent model for dealing with the variety of use cases in which dynptr is used.
First, for ARG_PTR_TO_DYNPTR, it can either be set alone or together with a DYNPTR_TYPE_* constant that denotes the only type it accepts.
Next, helpers which initialize a dynptr use MEM_UNINIT to indicate this fact. To make the distinction clear, use MEM_RDONLY flag to indicate that the helper only operates on the memory pointed to by the dynptr, not the dynptr itself. In C parlance, it would be equivalent to taking the dynptr as a point to const argument.
When either of these flags are not present, the helper is allowed to mutate both the dynptr itself and also the memory it points to. Currently, the read only status of the memory is not tracked in the dynptr, but it would be trivial to add this support inside dynptr state of the register.
With these changes and renaming PTR_TO_DYNPTR to CONST_PTR_TO_DYNPTR to better reflect its usage, it can no longer be passed to helpers that initialize a dynptr, i.e. bpf_dynptr_from_mem, bpf_ringbuf_reserve_dynptr.
A note to reviewers is that in code that does mark_stack_slots_dynptr, and unmark_stack_slots_dynptr, we implicitly rely on the fact that PTR_TO_STACK reg is the only case that can reach that code path, as one cannot pass CONST_PTR_TO_DYNPTR to helpers that don't set MEM_RDONLY. In both cases such helpers won't be setting that flag.
The next patch will add a couple of selftest cases to make sure this doesn't break.
--- include/linux/bpf.h | 4 +- include/uapi/linux/bpf.h | 8 +- kernel/bpf/helpers.c | 18 +- kernel/bpf/verifier.c | 227 +++++++++++++----- scripts/bpf_doc.py | 1 + tools/include/uapi/linux/bpf.h | 8 +- .../selftests/bpf/prog_tests/user_ringbuf.c | 4 +- 7 files changed, 191 insertions(+), 79 deletions(-)
include/linux/bpf.h +2 -2
@@ -775,7 +775,7 @@ enum bpf_reg_type {PTR_TO_MEM ,/* reg points to valid memory region */ PTR_TO_BUF ,/* reg points to a read/write buffer */ PTR_TO_FUNC ,/* reg points to a bpf program function */ -PTR_TO_DYNPTR ,/* reg points to a dynptr */ +CONST_PTR_TO_DYNPTR ,/* reg points to a const struct bpf_dynptr */ __BPF_REG_TYPE_MAX,/* Extended reg_types. */
@@ -2828,7 +2828,7 @@ void bpf_dynptr_init(struct bpf_dynptr_kern *ptr, void *data, enum bpf_dynptr_type type, u32 offset, u32 size);void bpf_dynptr_set_null (struct bpf_dynptr_kern *ptr);int bpf_dynptr_check_size (u32 size); -u32 bpf_dynptr_get_size (struct bpf_dynptr_kern *ptr); +u32 bpf_dynptr_get_size (const struct bpf_dynptr_kern *ptr);#ifdef CONFIG_BPF_LSM void bpf_cgroup_atype_get (u32 attach_btf_id,int cgroup_atype);
include/uapi/linux/bpf.h +4 -4
@@ -5293,7 +5293,7 @@ union bpf_attr {* Return * Nothing. Always succeeds. * -* long bpf_dynptr_read(void *dst, u32 len, struct bpf_dynptr *src, u32 offset, u64 flags) +* long bpf_dynptr_read(void *dst, u32 len, const struct bpf_dynptr *src, u32 offset, u64 flags) * Description * Read *len* bytes from *src* into *dst*, starting from *offset* * into *src*.
@@ -5303,7 +5303,7 @@ union bpf_attr {* of *src*'s data, -EINVAL if *src* is an invalid dynptr or if * *flags* is not 0. * -* long bpf_dynptr_write(struct bpf_dynptr *dst, u32 offset, void *src, u32 len, u64 flags) +* long bpf_dynptr_write(const struct bpf_dynptr *dst, u32 offset, void *src, u32 len, u64 flags) * Description * Write *len* bytes from *src* into *dst*, starting from *offset* * into *dst*.
@@ -5313,7 +5313,7 @@ union bpf_attr {* of *dst*'s data, -EINVAL if *dst* is an invalid dynptr or if *dst* * is a read-only dynptr or if *flags* is not 0. * -* void *bpf_dynptr_data(struct bpf_dynptr *ptr, u32 offset, u32 len) +* void *bpf_dynptr_data(const struct bpf_dynptr *ptr, u32 offset, u32 len) * Description * Get a pointer to the underlying dynptr data. *
@@ -5414,7 +5414,7 @@ union bpf_attr {* Drain samples from the specified user ring buffer, and invoke * the provided callback for each such sample: * -* long (\*callback_fn)(struct bpf_dynptr \*dynptr, void \*ctx); +* long (\*callback_fn)(const struct bpf_dynptr \*dynptr, void \*ctx); * * If **callback_fn** returns 0, the helper will continue to try * and drain the next sample, up to a maximum of
kernel/bpf/helpers.c +9 -9
@@ -1404,7 +1404,7 @@ static const struct bpf_func_proto bpf_kptr_xchg_proto = {#define DYNPTR_SIZE_MASK 0xFFFFFF#define DYNPTR_RDONLY_BIT BIT(31) -static bool bpf_dynptr_is_rdonly (struct bpf_dynptr_kern *ptr) +static bool bpf_dynptr_is_rdonly (const struct bpf_dynptr_kern *ptr) {return ptr->size &DYNPTR_RDONLY_BIT ; }
@@ -1414,7 +1414,7 @@ static void bpf_dynptr_set_type(struct bpf_dynptr_kern *ptr, enum bpf_dynptr_typ ptr->size |= type <<DYNPTR_TYPE_SHIFT ; } -u32 bpf_dynptr_get_size (struct bpf_dynptr_kern *ptr) +u32 bpf_dynptr_get_size (const struct bpf_dynptr_kern *ptr) {return ptr->size &DYNPTR_SIZE_MASK ; }
@@ -1438,7 +1438,7 @@ void bpf_dynptr_set_null(struct bpf_dynptr_kern *ptr)memset (ptr,0 ,sizeof (*ptr)); } -static int bpf_dynptr_check_off_len (struct bpf_dynptr_kern *ptr,u32 offset,u32 len) +static int bpf_dynptr_check_off_len (const struct bpf_dynptr_kern *ptr,u32 offset,u32 len) {u32 size =bpf_dynptr_get_size (ptr);
@@ -1483,7 +1483,7 @@ static const struct bpf_func_proto bpf_dynptr_from_mem_proto = { .arg4_type = ARG_PTR_TO_DYNPTR | DYNPTR_TYPE_LOCAL | MEM_UNINIT, }; -BPF_CALL_5(bpf_dynptr_read, void *, dst, u32, len, struct bpf_dynptr_kern *, src, +BPF_CALL_5(bpf_dynptr_read, void *, dst, u32, len, const struct bpf_dynptr_kern *, src, u32, offset, u64, flags) {int err;
@@ -1506,12 +1506,12 @@ static const struct bpf_func_proto bpf_dynptr_read_proto = { .ret_type = RET_INTEGER, .arg1_type = ARG_PTR_TO_UNINIT_MEM, .arg2_type = ARG_CONST_SIZE_OR_ZERO, - .arg3_type = ARG_PTR_TO_DYNPTR, + .arg3_type = ARG_PTR_TO_DYNPTR | MEM_RDONLY, .arg4_type = ARG_ANYTHING, .arg5_type = ARG_ANYTHING, }; -BPF_CALL_5(bpf_dynptr_write, struct bpf_dynptr_kern *, dst, u32, offset, void *, src, +BPF_CALL_5(bpf_dynptr_write, const struct bpf_dynptr_kern *, dst, u32, offset, void *, src, u32, len, u64, flags) {int err;
@@ -1532,14 +1532,14 @@ static const struct bpf_func_proto bpf_dynptr_write_proto = { .func = bpf_dynptr_write, .gpl_only = false, .ret_type = RET_INTEGER, - .arg1_type = ARG_PTR_TO_DYNPTR, + .arg1_type = ARG_PTR_TO_DYNPTR | MEM_RDONLY, .arg2_type = ARG_ANYTHING, .arg3_type = ARG_PTR_TO_MEM | MEM_RDONLY, .arg4_type = ARG_CONST_SIZE_OR_ZERO, .arg5_type = ARG_ANYTHING, }; -BPF_CALL_3(bpf_dynptr_data, struct bpf_dynptr_kern *, ptr, u32, offset, u32, len) +BPF_CALL_3(bpf_dynptr_data, const struct bpf_dynptr_kern *, ptr, u32, offset, u32, len) {int err;
@@ -1560,7 +1560,7 @@ static const struct bpf_func_proto bpf_dynptr_data_proto = { .func = bpf_dynptr_data, .gpl_only = false, .ret_type = RET_PTR_TO_DYNPTR_MEM_OR_NULL, - .arg1_type = ARG_PTR_TO_DYNPTR, + .arg1_type = ARG_PTR_TO_DYNPTR | MEM_RDONLY, .arg2_type = ARG_ANYTHING, .arg3_type = ARG_CONST_ALLOC_SIZE_OR_ZERO, };
kernel/bpf/verifier.c +169 -58
382 lines not on this page · raw message
scripts/bpf_doc.py +1 -0
@@ -752,6 +752,7 @@ class PrinterHelpers(Printer): 'struct bpf_timer', 'struct mptcp_sock', 'struct bpf_dynptr', + 'const struct bpf_dynptr', 'struct iphdr', 'struct ipv6hdr', }
tools/include/uapi/linux/bpf.h +4 -4
@@ -5293,7 +5293,7 @@ union bpf_attr {* Return * Nothing. Always succeeds. * -* long bpf_dynptr_read(void *dst, u32 len, struct bpf_dynptr *src, u32 offset, u64 flags) +* long bpf_dynptr_read(void *dst, u32 len, const struct bpf_dynptr *src, u32 offset, u64 flags) * Description * Read *len* bytes from *src* into *dst*, starting from *offset* * into *src*.
@@ -5303,7 +5303,7 @@ union bpf_attr {* of *src*'s data, -EINVAL if *src* is an invalid dynptr or if * *flags* is not 0. * -* long bpf_dynptr_write(struct bpf_dynptr *dst, u32 offset, void *src, u32 len, u64 flags) +* long bpf_dynptr_write(const struct bpf_dynptr *dst, u32 offset, void *src, u32 len, u64 flags) * Description * Write *len* bytes from *src* into *dst*, starting from *offset* * into *dst*.
@@ -5313,7 +5313,7 @@ union bpf_attr {* of *dst*'s data, -EINVAL if *dst* is an invalid dynptr or if *dst* * is a read-only dynptr or if *flags* is not 0. * -* void *bpf_dynptr_data(struct bpf_dynptr *ptr, u32 offset, u32 len) +* void *bpf_dynptr_data(const struct bpf_dynptr *ptr, u32 offset, u32 len) * Description * Get a pointer to the underlying dynptr data. *
@@ -5414,7 +5414,7 @@ union bpf_attr {* Drain samples from the specified user ring buffer, and invoke * the provided callback for each such sample: * -* long (\*callback_fn)(struct bpf_dynptr \*dynptr, void \*ctx); +* long (\*callback_fn)(const struct bpf_dynptr \*dynptr, void \*ctx); * * If **callback_fn** returns 0, the helper will continue to try * and drain the next sample, up to a maximum of
tools/testing/selftests/bpf/prog_tests/user_ringbuf.c +2 -2
@@ -673,8 +673,8 @@ static struct { {"user_ringbuf_callback_write_forbidden", "invalid mem access 'dynptr_ptr'"}, {"user_ringbuf_callback_null_context_write", "invalid mem access 'scalar'"}, {"user_ringbuf_callback_null_context_read", "invalid mem access 'scalar'"}, - {"user_ringbuf_callback_discard_dynptr", "arg 1 is an unacquired reference"}, - {"user_ringbuf_callback_submit_dynptr", "arg 1 is an unacquired reference"}, + {"user_ringbuf_callback_discard_dynptr", "cannot release unowned const bpf_dynptr"}, + {"user_ringbuf_callback_submit_dynptr", "cannot release unowned const bpf_dynptr"}, {"user_ringbuf_callback_invalid_return", "At callback return the register R0 has value"}, };
Signature · 1 line
2.38.1
Kumar Kartikeya Dwivedi
maintainerPATCH v2 4/7 bpf: Rework check_func_arg_reg_off
In reply to Kumar Kartikeya DwivediWhile check_func_arg_reg_off is the place which performs generic checks needed by various candidates of reg->type, there is some handling for special cases, like ARG_PTR_TO_DYNPTR, OBJ_RELEASE, and ARG_PTR_TO_ALLOC_MEM.
This commit aims to streamline these special cases and instead leave other things up to argument type specific code to handle. The function will be restrictive by default, and cover all possible cases when OBJ_RELEASE is set, without having to update the function again (and missing to do that being a bug).
This is done primarily for two reasons: associating back reg->type to its argument leaves room for the list getting out of sync when a new reg->type is supported by an arg_type.
The other case is ARG_PTR_TO_ALLOC_MEM. The problem there is something we already handle, whenever a release argument is expected, it should be passed as the pointer that was received from the acquire function. Hence zero fixed and variable offset.
There is nothing special about ARG_PTR_TO_ALLOC_MEM, where technically its target register type PTR_TO_MEM | MEM_ALLOC can already be passed with non-zero offset to other helper functions, which makes sense.
Hence, lift the arg_type_is_release check for reg->off and cover all possible register types, instead of duplicating the same kind of check twice for current OBJ_RELEASE arg_types (alloc_mem and ptr_to_btf_id).
For the release argument, arg_type_is_dynptr is the special case, where we go to actual object being freed through the dynptr, so the offset of the pointer still needs to allow fixed and variable offset and process_dynptr_func will verify them later for the release argument case as well.
This is not specific to ARG_PTR_TO_DYNPTR though, we will need to make this exception for any future object on the stack that needs to be released. In this sense, PTR_TO_STACK as a candidate for object on stack argument is a special case for release offset checks, and they need to be done by the helper releasing the object on stack.
Since the check has been lifted above all register type checks, remove the duplicated check that is being done for PTR_TO_BTF_ID.
--- kernel/bpf/verifier.c | 64 +++++++++++-------- tools/testing/selftests/bpf/verifier/calls.c | 2 +- .../testing/selftests/bpf/verifier/ringbuf.c | 2 +- 3 files changed, 41 insertions(+), 27 deletions(-)
kernel/bpf/verifier.c +39 -25
@@ -6269,11 +6269,38 @@ int check_func_arg_reg_off(struct bpf_verifier_env *env, const struct bpf_reg_state *reg, int regno, enum bpf_arg_type arg_type) { - enum bpf_reg_type type = reg->type; -bool fixed_off_ok = false; + u32 type = reg->type; -switch ((u32 )type) { -/* Pointer types where reg offset is explicitly allowed: */ +/* When referenced register is passed to release function, its fixed +* offset must be 0. +* +* We will check arg_type_is_release reg has ref_obj_id when storing +* meta->release_regno. +*/ +if (arg_type_is_release (arg_type)) { +/* ARG_PTR_TO_DYNPTR with OBJ_RELEASE is a bit special, as it +* may not directly point to the object being released, but to +* dynptr pointing to such object, which might be at some offset +* on the stack. In that case, we simply to fallback to the +* default handling. +*/ +if (!arg_type_is_dynptr (arg_type) || type !=PTR_TO_STACK ) { +/* Doing check_ptr_off_reg check for the offset will +* catch this because fixed_off_ok is false, but +* checking here allows us to give the user a better +* error message. +*/ +if (reg->off) { +verbose (env,"R%d must have zero offset when passed to release func or trusted arg to kfunc\n" , + regno); +return -EINVAL ; + } +return __check_ptr_off_reg (env, reg, regno, false); + } +/* Fallback to default handling */ + } +switch (type) { +/* Pointer types where both fixed and variable offset is explicitly allowed: */ case PTR_TO_STACK :if (arg_type_is_dynptr (arg_type) && reg->off %BPF_REG_SIZE ) {verbose (env,"cannot pass in dynptr at an offset\n" );
@@ -6290,12 +6317,7 @@ int check_func_arg_reg_off(struct bpf_verifier_env *env,case PTR_TO_BUF :case PTR_TO_BUF |MEM_RDONLY :case SCALAR_VALUE : -/* Some of the argument types nevertheless require a -* zero register offset. -*/ -if (base_type (arg_type) !=ARG_PTR_TO_RINGBUF_MEM ) -return 0 ; -break ; +return 0 ;/* All the rest must be rejected, except PTR_TO_BTF_ID which allows * fixed offset. */
@@ -6305,24 +6327,16 @@ int check_func_arg_reg_off(struct bpf_verifier_env *env,case PTR_TO_BTF_ID |MEM_RCU :case PTR_TO_BTF_ID |MEM_ALLOC |PTR_TRUSTED :/* When referenced PTR_TO_BTF_ID is passed to release function, -* it's fixed offset must be 0. In the other cases, fixed offset -* can be non-zero. +* its fixed offset must be 0. In the other cases, fixed offset +* can be non-zero. This was already checked above. So pass +* fixed_off_ok as true to allow fixed offset for all other +* cases. var_off always must be 0 for PTR_TO_BTF_ID, hence we +* still need to do checks instead of returning. */ -if (arg_type_is_release (arg_type) && reg->off) { -verbose (env,"R%d must have zero offset when passed to release func\n" , - regno); -return -EINVAL ; - } -/* For arg is release pointer, fixed_off_ok must be false, but -* we already checked and rejected reg->off != 0 above, so set -* to true to allow fixed offset for all other cases. -*/ - fixed_off_ok = true; -break ; +return __check_ptr_off_reg (env, reg, regno, true);default : -break ; +return __check_ptr_off_reg (env, reg, regno, false); } -return __check_ptr_off_reg (env, reg, regno, fixed_off_ok); }static u32 dynptr_ref_obj_id (struct bpf_verifier_env *env,struct bpf_reg_state *reg)
tools/testing/selftests/bpf/verifier/calls.c +1 -1
@@ -76,7 +76,7 @@ }, .prog_type = BPF_PROG_TYPE_SCHED_CLS, .result = REJECT, - .errstr = "arg#0 expected pointer to ctx, but got PTR", + .errstr = "R1 must have zero offset when passed to release func or trusted arg to kfunc", .fixup_kfunc_btf_id = { { "bpf_kfunc_call_test_pass_ctx", 2 }, },
tools/testing/selftests/bpf/verifier/ringbuf.c +1 -1
@@ -28,7 +28,7 @@ }, .fixup_map_ringbuf = { 1 }, .result = REJECT, - .errstr ="dereference of modified ringbuf_mem ptr R1" , + .errstr ="R1 must have zero offset when passed to release func" , }, { "ringbuf: invalid reservation offset 2",
Signature · 1 line
2.38.1
Re: [PATCH bpf-next v2 4/7] bpf: Rework check_func_arg_reg_off
Quoted 4 lines · Kumar Kartikeya Dwivedi
While
check_func_arg_reg_offis the place which performs generic checks needed by various candidates ofreg->type, there is some handling for special cases, likeARG_PTR_TO_DYNPTR,OBJ_RELEASE, andARG_PTR_TO_ALLOC_MEM.
commit log is obsolete.
I cleaned it up s/ARG_PTR_TO_ALLOC_MEM/ARG_PTR_TO_RINGBUF_MEM/
Quoted 10 lines
This commit aims to streamline these special cases and instead leave other things up to argument type specific code to handle. The function will be restrictive by default, and cover all possible cases when
OBJ_RELEASEis set, without having to update the function again (and missing to do that being a bug).This is done primarily for two reasons: associating back
reg->typeto its argument leaves room for the list getting out of sync when a newreg->typeis supported by anarg_type.
Quoted 7 lines
The other case is
ARG_PTR_TO_ALLOC_MEM. The problem there is something we already handle, whenever a release argument is expected, it should be passed as the pointer that was received from the acquire function. Hence zero fixed and variable offset.There is nothing special about
ARG_PTR_TO_ALLOC_MEM, where technically its target register typePTR_TO_MEM|MEM_ALLOCcan already be passed
and MEM_RINGBUF here.
Quoted 48 lines
with non-zero offset to other helper functions, which makes sense.
Hence, lift the
arg_type_is_releasecheck forreg->offand cover all possible register types, instead of duplicating the same kind of check twice for currentOBJ_RELEASEarg_types(alloc_memandptr_to_btf_id).For the release argument,
arg_type_is_dynptris the special case, where we go to actual object being freed through the dynptr, so the offset of the pointer still needs to allow fixed and variable offset andprocess_dynptr_funcwill verify them later for the release argument case as well.This is not specific to
ARG_PTR_TO_DYNPTRthough, we will need to make this exception for any future object on the stack that needs to be released. In this sense,PTR_TO_STACKas a candidate for object on stack argument is a special case for release offset checks, and they need to be done by the helper releasing the object on stack.Since the check has been lifted above all register type checks, remove the duplicated check that is being done for
PTR_TO_BTF_ID.Acked-by Joanne Koong <[email protected]>Acked-by David Vernet <[email protected]>Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>--- kernel/bpf/verifier.c | 64 +++++++++++-------- tools/testing/selftests/bpf/verifier/calls.c | 2 +- .../testing/selftests/bpf/verifier/ringbuf.c | 2 +- 3 files changed, 41 insertions(+), 27 deletions(-)kernel/bpf/verifier.c +5
-4@@ -6269,11 +6269,38 @@ int check_func_arg_reg_off(struct bpf_verifier_env *env, const struct bpf_reg_state *reg, int regno, enum bpf_arg_type arg_type) { - enum bpf_reg_type type = reg->type; -bool fixed_off_ok = false; + u32 type = reg->type; -switch ((u32 )type) { -/* Pointer types where reg offset is explicitly allowed: */ +/* When referenced register is passed to release function, its fixed +* offset must be 0. +* +* We will check arg_type_is_release reg has ref_obj_id when storing
kernel/bpf/verifier.c int check_func_arg_reg_off(struct bpf_verifier_env *env, · line 6287
Quoted 10 lines
kernel/bpf/verifier.c +10
-0+* meta->release_regno. +*/ +if (arg_type_is_release (arg_type)) { +/* ARG_PTR_TO_DYNPTR with OBJ_RELEASE is a bit special, as it +* may not directly point to the object being released, but to +* dynptr pointing to such object, which might be at some offset +* on the stack. In that case, we simply to fallback to the +* default handling. +*/ +if (!arg_type_is_dynptr (arg_type) || type !=PTR_TO_STACK ) {
also removed one indent, inverted above and added direct 'return 0' I know that it's not exactly the same, but together with patch 5 it makes it cleaner and this special case (that David didn't like) doesn't look as horrible anymore.
kernel/bpf/verifier.c int check_func_arg_reg_off(struct bpf_verifier_env *env, · line 6300
Quoted 13 lines
kernel/bpf/verifier.c +13
-0+/* Doing check_ptr_off_reg check for the offset will +* catch this because fixed_off_ok is false, but +* checking here allows us to give the user a better +* error message. +*/ +if (reg->off) { +verbose (env,"R%d must have zero offset when passed to release func or trusted arg to kfunc\n" , + regno); +return -EINVAL ; + } +return __check_ptr_off_reg (env, reg, regno, false); + } +/* Fallback to default handling */
No 'fallback to default' either.
It wasn't easy to follow this logic.
And applied.
See how it looks now.
Overall looks like great cleanup. Thanks!
Kumar Kartikeya Dwivedi
maintainerPATCH v2 5/7 bpf: Move PTR_TO_STACK alignment check to process_dynptr_func
In reply to Kumar Kartikeya DwivediAfter previous commit, we are minimizing helper specific assumptions from check_func_arg_reg_off, making it generic, and offloading checks for a specific argument type to their respective functions called after check_func_arg_reg_off has been called.
This allows relying on a consistent set of guarantees after that call and then relying on them in code that deals with registers for each argument type later. This is in line with how process_spin_lock, process_timer_func, process_kptr_func check reg->var_off to be constant. The same reasoning is used here to move the alignment check into process_dynptr_func. Note that it also needs to check for constant var_off, and accumulate the constant var_off when computing the spi in get_spi, but that fix will come in later changes.
--- kernel/bpf/verifier.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-)
kernel/bpf/verifier.c +8 -5
@@ -5932,6 +5932,14 @@ int process_dynptr_func(struct bpf_verifier_env *env, int regno,verbose (env,"verifier internal error: misconfigured dynptr helper type flags\n" );return -EFAULT ; } +/* CONST_PTR_TO_DYNPTR already has fixed and var_off as 0 due to +* check_func_arg_reg_off's logic. We only need to check offset +* alignment for PTR_TO_STACK. +*/ +if (reg->type ==PTR_TO_STACK && (reg->off %BPF_REG_SIZE )) { +verbose (env,"cannot pass in dynptr at an offset=%d\n" , reg->off); +return -EINVAL ; + }/* MEM_UNINIT - Points to memory that is an appropriate candidate for * constructing a mutable bpf_dynptr object. *
@@ -6302,11 +6310,6 @@ int check_func_arg_reg_off(struct bpf_verifier_env *env,switch (type) {/* Pointer types where both fixed and variable offset is explicitly allowed: */ case PTR_TO_STACK : -if (arg_type_is_dynptr (arg_type) && reg->off %BPF_REG_SIZE ) { -verbose (env,"cannot pass in dynptr at an offset\n" ); -return -EINVAL ; - } - fallthrough;case PTR_TO_PACKET :case PTR_TO_PACKET_META :case PTR_TO_MAP_KEY :
Signature · 1 line
2.38.1
Kumar Kartikeya Dwivedi
maintainerPATCH v2 6/7 bpf: Use memmove for bpf_dynptr_{read,write}
In reply to Kumar Kartikeya DwivediIt may happen that destination buffer memory overlaps with memory dynptr points to. Hence, we must use memmove to correctly copy from dynptr to destination buffer, or source buffer to dynptr.
This actually isn't a problem right now, as memcpy implementation falls back to memmove on detecting overlap and warns about it, but we shouldn't be relying on that.
--- kernel/bpf/helpers.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-)
kernel/bpf/helpers.c +10 -2
@@ -1495,7 +1495,11 @@ BPF_CALL_5(bpf_dynptr_read, void *, dst, u32, len, const struct bpf_dynptr_kernif (err)return err; -memcpy (dst, src->data + src->offset + offset, len); +/* Source and destination may possibly overlap, hence use memmove to +* copy the data. E.g. bpf_dynptr_from_mem may create two dynptr +* pointing to overlapping PTR_TO_MAP_VALUE regions. +*/ +memmove (dst, src->data + src->offset + offset, len);return 0 ; }
@@ -1523,7 +1527,11 @@ BPF_CALL_5(bpf_dynptr_write, const struct bpf_dynptr_kern *, dst, u32, offset, vif (err)return err; -memcpy (dst->data + dst->offset + offset, src, len); +/* Source and destination may possibly overlap, hence use memmove to +* copy the data. E.g. bpf_dynptr_from_mem may create two dynptr +* pointing to overlapping PTR_TO_MAP_VALUE regions. +*/ +memmove (dst->data + dst->offset + offset, src, len);return 0 ; }
Signature · 1 line
2.38.1
Kumar Kartikeya Dwivedi
maintainerPATCH v2 7/7 selftests/bpf: Add test for dynptr reinit in user_ringbuf callback
In reply to Kumar Kartikeya DwivediThe original support for bpf_user_ringbuf_drain callbacks simply short-circuited checks for the dynptr state, allowing users to pass PTR_TO_DYNPTR (now CONST_PTR_TO_DYNPTR) to helpers that initialize a dynptr. This bug would have also surfaced with other dynptr helpers in the future that changed dynptr view or modified it in some way.
Include test cases for all cases, i.e. both bpf_dynptr_from_mem and bpf_ringbuf_reserve_dynptr, and ensure verifier rejects both of them. Without the fix, both of these programs load and pass verification.
While at it, remove sys_nanosleep target from failure cases' SEC definition, as there is no such tracepoint.
--- .../selftests/bpf/prog_tests/user_ringbuf.c | 2 + .../selftests/bpf/progs/user_ringbuf_fail.c | 51 ++++++++++++++++--- 2 files changed, 45 insertions(+), 8 deletions(-)
tools/testing/selftests/bpf/prog_tests/user_ringbuf.c +2 -0
@@ -676,6 +676,8 @@ static struct { {"user_ringbuf_callback_discard_dynptr", "cannot release unowned const bpf_dynptr"}, {"user_ringbuf_callback_submit_dynptr", "cannot release unowned const bpf_dynptr"}, {"user_ringbuf_callback_invalid_return", "At callback return the register R0 has value"}, + {"user_ringbuf_callback_reinit_dynptr_mem", "Dynptr has to be an uninitialized dynptr"}, + {"user_ringbuf_callback_reinit_dynptr_ringbuf", "Dynptr has to be an uninitialized dynptr"}, };#define SUCCESS_TEST (_func) { _func, #_func }
tools/testing/selftests/bpf/progs/user_ringbuf_fail.c +43 -8
108 lines not on this page · raw message
Signature · 1 line
2.38.1
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Hello:
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Quoted 10 lines · On Thu, 8 Dec 2022 02:11:34 +0530 you wrote:
This is part 1 of https://lore.kernel.org/bpf/[email protected].
This thread also gives some background on why the refactor is being done:
https://lore.kernel.org/bpf/CAEf4Bzb4beTHgVo+G+jehSj8oCeAjRbRcm6MRe=Gr+cajRBwEw@mail.gmail.comAs requested in patch 6 by Alexei, it only includes patches which refactors the code, on top of which further fixes will be made in part
2. The refactor itself fixes another issue as a side effect. No functional change is intended (except a few modified log messages).[...]
Here is the summary with links:
- [bpf-next,v2,1/7] bpf: Refactor ARG_PTR_TO_DYNPTR checks into process_dynptr_func
https://git.kernel.org/bpf/bpf-next/c/6b75bd3d0367 - [bpf-next,v2,2/7] bpf: Propagate errors from process_* checks in check_func_arg https://git.kernel.org/bpf/bpf-next/c/ac50fe51ce87 - [bpf-next,v2,3/7] bpf: Rework process_dynptr_func https://git.kernel.org/bpf/bpf-next/c/270605317366 - [bpf-next,v2,4/7] bpf: Rework check_func_arg_reg_off https://git.kernel.org/bpf/bpf-next/c/184c9bdb8f65 - [bpf-next,v2,5/7] bpf: Move PTR_TO_STACK alignment check to process_dynptr_func https://git.kernel.org/bpf/bpf-next/c/f6ee298fa140 - [bpf-next,v2,6/7] bpf: Use memmove for bpf_dynptr_{read,write} https://git.kernel.org/bpf/bpf-next/c/76d16077bef0 - [bpf-next,v2,7/7] selftests/bpf: Add test for dynptr reinit in user_ringbuf callback https://git.kernel.org/bpf/bpf-next/c/292064cce796
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html