Support storing struct task_struct objects as kptrs
Subsystems (4)
v3 cover Support storing struct task_struct objects as kptrs
Now that BPF supports adding new kernel functions with kfuncs, and storing kernel objects in maps with kptrs, we can add a set of kfuncs which allow struct task_struct objects to be stored in maps as referenced kptrs.
The possible use cases for doing this are plentiful. During tracing, for example, it would be useful to be able to collect some tasks that performed a certain operation, and then periodically summarize who they are, which cgroup they're in, how much CPU time they've utilized, etc. Doing this now would require storing the task's pids along with some relevant data to be exported to user space, and later associating the pids to tasks in other event handlers where the data is recorded. Another useful by-product of this is that it allows a program to pin a task in a BPF program, and by proxy therefore also e.g. pin its task local storage.
In order to support this, we'll need to expand KF_TRUSTED_ARGS to support receiving trusted, non-refcounted pointers. It currently only supports either PTR_TO_CTX pointers, or refcounted pointers . What this means in terms of implementation is that btf_check_func_arg_match() would have to add another condition to its logic for checking if a ptr needs a refcount to also require that the pointer has at least one type modifier, such as PTR_UNTRUSTED. PTR_UNTRUSTED does not cover all of the possible pointers we need to watch out for, though. For example, a pointer obtained from walking a struct is considered "trusted" (or at least, not PTR_UNTRUSTED). To account for this and enable us to expand KF_TRUSTED_ARGS, this patch set also introduces a new PTR_NESTED type flag modifier which records if a pointer was obtained from walking a struct.
This patch set adds this new PTR_NESTED type flag, expands KF_TRUSTED_ARGS accordingly, adds the new set of kfuncs mentioned above, and then finally adds a new selftest suite to validate all of this new behavior.
David Vernet (3):
bpf: Allow trusted pointers to be passed to KF_TRUSTED_ARGS kfuncs
bpf: Add kfuncs for storing struct task_struct * as a kptr bpf/selftests: Add selftests for new task kfuncs
include/linux/bpf.h | 6 + kernel/bpf/btf.c | 11 +- kernel/bpf/helpers.c | 86 ++++- kernel/bpf/verifier.c | 15 +- tools/testing/selftests/bpf/DENYLIST.s390x | 1 + .../selftests/bpf/prog_tests/task_kfunc.c | 160 +++++++++ .../selftests/bpf/progs/task_kfunc_common.h | 83 +++++ .../selftests/bpf/progs/task_kfunc_failure.c | 315 ++++++++++++++++++ .../selftests/bpf/progs/task_kfunc_success.c | 132 ++++++++ tools/testing/selftests/bpf/verifier/calls.c | 4 +- 10 files changed, 804 insertions(+), 9 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/task_kfunc.c create mode 100644 tools/testing/selftests/bpf/progs/task_kfunc_common.h create mode 100644 tools/testing/selftests/bpf/progs/task_kfunc_failure.c create mode 100644 tools/testing/selftests/bpf/progs/task_kfunc_success.c
Signature · 1 line
2.38.0
PATCH v3 2/3 bpf: Add kfuncs for storing struct task_struct * as a kptr
Now that BPF supports adding new kernel functions with kfuncs, and storing kernel objects in maps with kptrs, we can add a set of kfuncs which allow struct task_struct objects to be stored in maps as referenced kptrs. The possible use cases for doing this are plentiful. During tracing, for example, it would be useful to be able to collect some tasks that performed a certain operation, and then periodically summarize who they are, which cgroup they're in, how much CPU time they've utilized, etc.
In order to enable this, this patch adds three new kfuncs:
struct task_struct *bpf_task_acquire(struct task_struct *p); struct task_struct *bpf_task_kptr_get(struct task_struct **pp); void bpf_task_release(struct task_struct *p);
A follow-on patch will add selftests validating these kfuncs.
--- kernel/bpf/helpers.c | 86 +++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 81 insertions(+), 5 deletions(-)
kernel/bpf/helpers.c +81 -5
@@ -1700,20 +1700,96 @@ bpf_base_func_proto(enum bpf_func_id func_id) } } -BTF_SET8_START(tracing_btf_ids) +__diag_push (); +__diag_ignore_all ("-Wmissing-prototypes" , +"Global functions as their definitions will be in vmlinux BTF" ); + +/** +* bpf_task_acquire - Acquire a reference to a task. A task acquired by this +* kfunc which is not stored in a map as a kptr, must be released by calling +* bpf_task_release(). +* @p: The task on which a reference is being acquired. +*/ +__used noinline +struct task_struct *bpf_task_acquire(struct task_struct *p) +{ +if (!p) +return NULL ; + +refcount_inc (&p->rcu_users); +return p; +} + +/** +* bpf_task_kptr_get - Acquire a reference on a struct task_struct kptr. A task +* kptr acquired by this kfunc which is not subsequently stored in a map, must +* be released by calling bpf_task_release(). +* @pp: A pointer to a task kptr on which a reference is being acquired. +*/ +__used noinline +struct task_struct *bpf_task_kptr_get(struct task_struct **pp) +{ +struct task_struct *p; + +rcu_read_lock (); + p =READ_ONCE (*pp); +if (p && !refcount_inc_not_zero (&p->rcu_users)) + p =NULL ; +rcu_read_unlock (); + +return p; +} + +/** +* bpf_task_release - Release the reference acquired on a struct task_struct *. +* If this kfunc is invoked in an RCU read region, the task_struct is +* guaranteed to not be freed until the current grace period has ended, even if +* its refcount drops to 0. +* @p: The task on which a reference is being released. +*/ +__used noinline void bpf_task_release(struct task_struct *p) +{ +if (!p) +return ; + +put_task_struct_rcu_user (p); +} + +__diag_pop (); + +BTF_SET8_START(generic_kfunc_btf_ids)#ifdef CONFIG_KEXEC_CORE BTF_ID_FLAGS(func, crash_kexec, KF_DESTRUCTIVE)#endif -BTF_SET8_END (tracing_btf_ids) +BTF_ID_FLAGS(func, bpf_task_acquire, KF_ACQUIRE | KF_RET_NULL | KF_TRUSTED_ARGS) +BTF_ID_FLAGS(func, bpf_task_kptr_get, KF_ACQUIRE | KF_KPTR_GET | KF_RET_NULL) +BTF_ID_FLAGS(func, bpf_task_release, KF_RELEASE | KF_TRUSTED_ARGS) +BTF_SET8_END (generic_kfunc_btf_ids) -static const struct btf_kfunc_id_set tracing_kfunc_set = { +static const struct btf_kfunc_id_set generic_kfunc_set = { .owner =THIS_MODULE , - .set = &tracing_btf_ids, + .set = &generic_kfunc_btf_ids, }; +BTF_ID_LIST (generic_kfunc_dtor_ids) +BTF_ID(struct, task_struct) +BTF_ID (func, bpf_task_release) + static int__init kfunc_init (void) { -return register_btf_kfunc_id_set (BPF_PROG_TYPE_TRACING , &tracing_kfunc_set); +int ret; +const struct btf_id_dtor_kfunc generic_kfunc_dtors[] = { + { + .btf_id = generic_kfunc_dtor_ids[0 ], + .kfunc_btf_id = generic_kfunc_dtor_ids[1 ] + }, + }; + + ret =register_btf_kfunc_id_set (BPF_PROG_TYPE_TRACING , &generic_kfunc_set); + ret = ret ?:register_btf_kfunc_id_set (BPF_PROG_TYPE_STRUCT_OPS , &generic_kfunc_set); +return ret ?:register_btf_id_dtor_kfuncs (generic_kfunc_dtors, +ARRAY_SIZE (generic_kfunc_dtors), +THIS_MODULE ); }late_initcall (kfunc_init);
Signature · 1 line
2.38.0
PATCH v3 1/3 bpf: Allow trusted pointers to be passed to KF_TRUSTED_ARGS kfuncs
In reply to David VernetKfuncs currently support specifying the KF_TRUSTED_ARGS flag to signal to the verifier that it should enforce that a BPF program passes it a "safe", trusted pointer. Currently, "safe" means that the pointer is either PTR_TO_CTX, or is refcounted. There may be cases, however, where the kernel passes a BPF program a safe / trusted pointer to an object that the BPF program wishes to use as a kptr, but because the object does not yet have a ref_obj_id from the perspective of the verifier, the program would be unable to pass it to a KF_ACQUIRE | KF_TRUSTED_ARGS kfunc.
The solution is to expand the set of pointers that are considered trusted according to KF_TRUSTED_ARGS, so that programs can invoke kfuncs with these pointers without getting rejected by the verifier.
There is already a PTR_UNTRUSTED flag that is set in some scenarios, such as when a BPF program reads a kptr directly from a map without performing a bpf_kptr_xchg() call. These pointers of course can and should be rejected by the verifier. Unfortunately, however, PTR_UNTRUSTED does not cover all the cases for safety that need to be addressed to adequately protect kfuncs. Specifically, pointers obtained by a BPF program "walking" a struct are _not_ considered PTR_UNTRUSTED according to BPF. For example, say that we were to add a kfunc called bpf_task_acquire(), with KF_ACQUIRE | KF_TRUSTED_ARGS, to acquire a struct task_struct *. If we only used PTR_UNTRUSTED to signal that a task was unsafe to pass to a kfunc, the verifier would mistakenly allow the following unsafe BPF program to be loaded:
SEC("tp_btf/task_newtask")
int BPF_PROG(unsafe_acquire_task,
struct task_struct *task,
u64 clone_flags)
{
struct task_struct *acquired, *nested;
nested = task->last_wakee;
/* Would not be rejected by the verifier. */
acquired = bpf_task_acquire(nested);
if (!acquired)
return 0;
bpf_task_release(acquired);
return 0;
}To address this, this patch defines a new type flag called PTR_NESTED which tracks whether a PTR_TO_BTF_ID pointer was retrieved from walking a struct. A pointer passed directly from the kernel begins with (PTR_NESTED & type) == 0, meaning of course that it is not nested. Any pointer received from walking that object, however, would inherit that flag and become a nested pointer.
With that flag, this patch also updates btf_check_func_arg_match() to only flag a PTR_TO_BTF_ID object as requiring a refcount if it has any type modifiers (which of course includes both PTR_UNTRUSTED and PTR_NESTED). Otherwise, the pointer passes this check and continues onto the others in btf_check_func_arg_match().
A subsequent patch will add kfuncs for storing a task kfunc as a kptr, and then another patch will validate this feature by ensuring that the verifier rejects a kfunc invocation with a nested pointer.
--- include/linux/bpf.h | 6 ++++++ kernel/bpf/btf.c | 11 ++++++++++- kernel/bpf/verifier.c | 15 ++++++++++++++- tools/testing/selftests/bpf/verifier/calls.c | 4 ++-- 4 files changed, 32 insertions(+), 4 deletions(-)
include/linux/bpf.h +6 -0
@@ -457,6 +457,12 @@ enum bpf_type_flag {/* Size is known at compile time. */ MEM_FIXED_SIZE =BIT (10 +BPF_BASE_TYPE_BITS ), +/* PTR was obtained from walking a struct. This is used with +* PTR_TO_BTF_ID to determine whether the pointer is safe to pass to a +* kfunc with KF_TRUSTED_ARGS. +*/ +PTR_NESTED =BIT (11 +BPF_BASE_TYPE_BITS ), + __BPF_TYPE_FLAG_MAX, __BPF_TYPE_LAST_FLAG = __BPF_TYPE_FLAG_MAX - 1, };
kernel/bpf/btf.c +10 -1
@@ -6333,8 +6333,17 @@ static int btf_check_func_arg_match(struct bpf_verifier_env *env,/* Check if argument must be a referenced pointer, args + i has * been verified to be a pointer (after skipping modifiers). * PTR_TO_CTX is ok without having non-zero ref_obj_id. +* +* All object pointers must be refcounted, other than: +* - PTR_TO_CTX +* - Trusted pointers (i.e. pointers with no type modifiers) */ -if (is_kfunc && trusted_args && (obj_ptr && reg->type !=PTR_TO_CTX ) && !reg->ref_obj_id) { +if (is_kfunc && + trusted_args && + obj_ptr && +base_type (reg->type) !=PTR_TO_CTX && +type_flag (reg->type) && + !reg->ref_obj_id) {bpf_log (log,"R%d must be referenced\n" , regno);return -EINVAL ; }
kernel/bpf/verifier.c +14 -1
@@ -581,6 +581,8 @@ static const char *reg_type_str(struct bpf_verifier_env *env,strncpy (prefix,"user_" ,32 );if (type &MEM_PERCPU )strncpy (prefix,"percpu_" ,32 ); +if (type &PTR_NESTED ) +strncpy (prefix,"nested_" ,32 );if (type &PTR_UNTRUSTED )strncpy (prefix,"untrusted_" ,32 );
@@ -4558,6 +4560,9 @@ static int check_ptr_to_btf_access(struct bpf_verifier_env *env,if (type_flag (reg->type) &PTR_UNTRUSTED ) flag |=PTR_UNTRUSTED ; +/* A pointer can only be unwalked when it wasn't accessed by walking a struct. */ + flag |=PTR_NESTED ; +if (atype ==BPF_READ && value_regno >=0 )mark_btf_ld_reg (env, regs, value_regno, ret, reg->btf, btf_id, flag);
@@ -5694,7 +5699,12 @@ static const struct bpf_reg_types scalar_types = { .types = { SCALAR_VALUE } };static const struct bpf_reg_types context_types = { .types = {PTR_TO_CTX } };static const struct bpf_reg_types alloc_mem_types = { .types = {PTR_TO_MEM |MEM_ALLOC } };static const struct bpf_reg_types const_map_ptr_types = { .types = {CONST_PTR_TO_MAP } }; -static const struct bpf_reg_types btf_ptr_types = { .types = {PTR_TO_BTF_ID } }; +static const struct bpf_reg_types btf_ptr_types = { + .types = { +PTR_TO_BTF_ID , +PTR_TO_BTF_ID |PTR_NESTED + }, +};static const struct bpf_reg_types spin_lock_types = { .types = {PTR_TO_MAP_VALUE } };static const struct bpf_reg_types percpu_btf_ptr_types = { .types = {PTR_TO_BTF_ID |MEM_PERCPU } };static const struct bpf_reg_types func_ptr_types = { .types = {PTR_TO_FUNC } };
@@ -5768,6 +5778,9 @@ static int check_reg_type(struct bpf_verifier_env *env, u32 regno,if (arg_type &PTR_MAYBE_NULL ) type &= ~PTR_MAYBE_NULL ; +if (!(arg_type &ARG_PTR_TO_BTF_ID )) + type &= ~PTR_NESTED ; +for (i =0 ; i <ARRAY_SIZE (compatible->types); i++) { expected = compatible->types[i];if (expected ==NOT_INIT )
tools/testing/selftests/bpf/verifier/calls.c +2 -2
@@ -181,7 +181,7 @@ }, .result_unpriv = REJECT, .result = REJECT, - .errstr = "negative offset ptr_ ptr R1 off=-4 disallowed", + .errstr = "negative offset nested_ptr_ ptr R1 off=-4 disallowed", }, { "calls: invalid kfunc call: PTR_TO_BTF_ID with variable offset",
@@ -243,7 +243,7 @@ }, .result_unpriv = REJECT, .result = REJECT, - .errstr = "R1 must be referenced", + .errstr = "arg#0 pointer type STRUCT prog_test_ref_kfunc must point to scalar", }, { "calls: valid kfunc call: referenced arg needs refcounted PTR_TO_BTF_ID",
Signature · 1 line
2.38.0
PATCH v3 3/3 bpf/selftests: Add selftests for new task kfuncs
In reply to David VernetA previous change added a series of kfuncs for storing struct task_struct objects as referenced kptrs. This patch adds a new task_kfunc test suite for validating their expected behavior.
--- tools/testing/selftests/bpf/DENYLIST.s390x | 1 + .../selftests/bpf/prog_tests/task_kfunc.c | 160 +++++++++ .../selftests/bpf/progs/task_kfunc_common.h | 83 +++++ .../selftests/bpf/progs/task_kfunc_failure.c | 315 ++++++++++++++++++ .../selftests/bpf/progs/task_kfunc_success.c | 132 ++++++++ 5 files changed, 691 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/task_kfunc.c create mode 100644 tools/testing/selftests/bpf/progs/task_kfunc_common.h create mode 100644 tools/testing/selftests/bpf/progs/task_kfunc_failure.c create mode 100644 tools/testing/selftests/bpf/progs/task_kfunc_success.c
tools/testing/selftests/bpf/DENYLIST.s390x +1 -0
@@ -52,6 +52,7 @@ skc_to_unix_sock # could not attach BPF object unexpecte socket_cookie # prog_attach unexpected error: -524 (trampoline) stacktrace_build_id # compare_map_keys stackid_hmap vs. stackmap err -2 errno 2 (?) tailcalls # tail_calls are not allowed in non-JITed programs with bpf-to-bpf calls (?) +task_kfunc # JIT does not support calling kernel function task_local_storage # failed to auto-attach program 'trace_exit_creds': -524 (trampoline) test_bpffs # bpffs test failed 255 (iterator) test_bprm_opts # failed to auto-attach program 'secure_exec': -524 (trampoline)
tools/testing/selftests/bpf/prog_tests/task_kfunc.c new file +160 -0
160 lines not on this page · raw message
tools/testing/selftests/bpf/progs/task_kfunc_common.h new file +83 -0
83 lines not on this page · raw message
tools/testing/selftests/bpf/progs/task_kfunc_failure.c new file +315 -0
315 lines not on this page · raw message
tools/testing/selftests/bpf/progs/task_kfunc_success.c new file +132 -0
132 lines not on this page · raw message
Signature · 1 line
2.38.0
Re: [PATCH v3 1/3] bpf: Allow trusted pointers to be passed to KF_TRUSTED_ARGS kfuncs
In reply to David VernetOn Fri, Oct 14, 2022 at 03:14:25PM -0500, David Vernet wrote:
[...]
kernel/bpf/verifier.c static int check_reg_type(struct bpf_verifier_env *env, u32 regno, · line 5783
Quoted 7 lines
kernel/bpf/verifier.c +3
-0@@ -5768,6 +5778,9 @@ static int check_reg_type(struct bpf_verifier_env *env, u32 regno,if (arg_type &PTR_MAYBE_NULL ) type &= ~PTR_MAYBE_NULL ; +if (!(arg_type &ARG_PTR_TO_BTF_ID )) + type &= ~PTR_NESTED ; +
Argh, I added while debugging something and I meant to remove it before sending out the next version. I'll send out v4 shortly without this, apologies for the noise.
Kumar Kartikeya Dwivedi
maintainerRe: [PATCH v3 3/3] bpf/selftests: Add selftests for new task kfuncs
In reply to David VernetQuoted 9 lines · David Vernet
A previous change added a series of kfuncs for storing struct
task_structobjects as referenced kptrs. This patch adds a newtask_kfunctest suite for validating their expected behavior.Signed-off-by David Vernet <[email protected]>---
[...]tools/testing/selftests/bpf/progs/task_kfunc_failure.c new file +1
-0+
tools/testing/selftests/bpf/progs/task_kfunc_failure.c line 93
Quoted 10 lines
tools/testing/selftests/bpf/progs/task_kfunc_failure.c new file +10
-0+SEC("tp_btf/task_newtask") +int BPF_PROG (task_kfunc_acquire_trusted_nested ,struct task_struct *task,u64 clone_flags) +{ +struct task_struct *acquired; + +if (!is_test_kfunc_task ()) +return 0 ; + +/* Can't invoke bpf_task_acquire() on a trusted pointer at a nonzero offset. */ + acquired =bpf_task_acquire (task->last_wakee);
The comment is incorrect, that would be &task->last_wakee instead, this is PTR_TO_BTF_ID | PTR_NESTED.
Quoted 4 lines
tools/testing/selftests/bpf/progs/task_kfunc_failure.c new file +4
-0+if (!acquired) +return 0 ; +bpf_task_release (acquired); +
tools/testing/selftests/bpf/progs/task_kfunc_failure.c line 77
Quoted 10 lines
tools/testing/selftests/bpf/progs/task_kfunc_failure.c new file +9
-0+return 0 ; +} + + +static int test_acquire_release (struct task_struct *task) +{ +struct task_struct *acquired; + + acquired =bpf_task_acquire (task);
Unfortunately a side effect of this change is that now since PTR_TO_BTF_ID without ref_obj_id is considered trusted, the bpf_ct_* functions would begin working with tp_btf args. That probably needs to be fixed so that they reject them (ideally with a failing test case to make sure it doesn't resurface), probably with a new suffix __ref/or __owned as added here [0].
Alexei, since you've suggested avoiding adding that suffix, do you see any other way out here?
It's questionable whether bpf_ct_set_timeout/status should work for CT not owned by the BPF program.
Quoted 11 lines
tools/testing/selftests/bpf/progs/task_kfunc_success.c new file +10
-0+if (!acquired) { + err =1 ; +return 0 ; + } + +bpf_task_release (acquired); + +return 0 ; +} +
Kumar Kartikeya Dwivedi
maintainerRe: [PATCH v3 3/3] bpf/selftests: Add selftests for new task kfuncs
Quoted 47 lines · Kumar Kartikeya Dwivedi
Quoted 9 lines · David Vernet
A previous change added a series of kfuncs for storing struct
task_structobjects as referenced kptrs. This patch adds a newtask_kfunctest suite for validating their expected behavior.Signed-off-by David Vernet <[email protected]>---
[...]tools/testing/selftests/bpf/progs/task_kfunc_failure.c new file +1
-0+tools/testing/selftests/bpf/progs/task_kfunc_failure.c line 93
Quoted 10 lines
tools/testing/selftests/bpf/progs/task_kfunc_failure.c new file +10
-0+SEC("tp_btf/task_newtask") +int BPF_PROG (task_kfunc_acquire_trusted_nested ,struct task_struct *task,u64 clone_flags) +{ +struct task_struct *acquired; + +if (!is_test_kfunc_task ()) +return 0 ; + +/* Can't invoke bpf_task_acquire() on a trusted pointer at a nonzero offset. */ + acquired =bpf_task_acquire (task->last_wakee);The comment is incorrect, that would be
&task->last_wakeeinstead, this isPTR_TO_BTF_ID|PTR_NESTED.Quoted 4 lines
tools/testing/selftests/bpf/progs/task_kfunc_failure.c new file +4
-0+if (!acquired) +return 0 ; +bpf_task_release (acquired); +tools/testing/selftests/bpf/progs/task_kfunc_failure.c line 77
Quoted 10 lines
tools/testing/selftests/bpf/progs/task_kfunc_failure.c new file +9
-0+return 0 ; +} + + +static int test_acquire_release (struct task_struct *task) +{ +struct task_struct *acquired; + + acquired =bpf_task_acquire (task);Unfortunately a side effect of this change is that now since
PTR_TO_BTF_IDwithoutref_obj_idis considered trusted, the bpf_ct_* functions would begin working withtp_btfargs. That probably needs to be fixed so that they reject them (ideally with a failing test case to make sure it doesn't resurface), probably with a new suffix __ref/or__ownedas added here [0].
Quoted 7 lines
Alexei, since you've suggested avoiding adding that suffix, do you see any other way out here?
It's questionable whether bpf_ct_set_timeout/status should work for CT not owned by the BPF program.
Sorry for accidentally bumping this v3 thread instead of v5, whosoever replies please continue there.