bpf: return PTR_TO_BTF_ID | PTR_TRUSTED from BPF kfuncs by default
Subsystems (6)
PATCH 1/3 bpf: return PTR_TO_BTF_ID | PTR_TRUSTED from BPF kfuncs by default
Teach the BPF verifier to treat pointers to struct types returned from BPF kfuncs as implicitly trusted (PTR_TO_BTF_ID | PTR_TRUSTED) by default. Returning untrusted pointers to struct types from BPF kfuncs should be considered an exception only, and certainly not the norm.
Update existing selftests to reflect the change in register type printing (e.g. ptr_ becoming trusted_ptr_ in verifier error messages).
--- kernel/bpf/verifier.c | 46 ++++++++++++------- .../selftests/bpf/progs/map_kptr_fail.c | 4 +- .../struct_ops_kptr_return_fail__wrong_type.c | 2 +- .../bpf/progs/verifier_global_ptr_args.c | 2 +- tools/testing/selftests/bpf/verifier/calls.c | 2 +- 5 files changed, 34 insertions(+), 22 deletions(-)
kernel/bpf/verifier.c +29 -17
@@ -14216,26 +14216,38 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,if (is_kfunc_rcu_protected (&meta)) regs[BPF_REG_0 ].type |=MEM_RCU ; }else { -mark_reg_known_zero (env, regs,BPF_REG_0 ); - regs[BPF_REG_0 ].btf = desc_btf; - regs[BPF_REG_0 ].type =PTR_TO_BTF_ID ; - regs[BPF_REG_0 ].btf_id = ptr_type_id; +enum bpf_reg_type type =PTR_TO_BTF_ID ;if (meta.func_id == special_kfunc_list[KF_bpf_get_kmem_cache]) - regs[BPF_REG_0 ].type |=PTR_UNTRUSTED ; -else if (is_kfunc_rcu_protected (&meta)) - regs[BPF_REG_0 ].type |=MEM_RCU ; - -if (is_iter_next_kfunc (&meta)) { -struct bpf_reg_state *cur_iter; - - cur_iter =get_iter_from_state (env->cur_state, &meta); - -if (cur_iter->type &MEM_RCU )/* KF_RCU_PROTECTED */ - regs[BPF_REG_0 ].type |=MEM_RCU ; -else - regs[BPF_REG_0 ].type |=PTR_TRUSTED ; + type |=PTR_UNTRUSTED ; +else if (is_kfunc_rcu_protected (&meta) || + (is_iter_next_kfunc (&meta) && + (get_iter_from_state (env->cur_state, &meta) + ->type &MEM_RCU ))) { +/* +* If the iterator's constructor (the _new +* function e.g., bpf_iter_task_new) has been +* annotated with BPF kfunc flag +* KF_RCU_PROTECTED and was called within a RCU +* read-side critical section, also propagate +* the MEM_RCU flag to the pointer returned from +* the iterator's next function (e.g., +* bpf_iter_task_next). +*/ + type |=MEM_RCU ; + }else { +/* +* Any PTR_TO_BTF_ID that is returned from a BPF +* kfunc should by default be treated as +* implicitly trusted. +*/ + type |=PTR_TRUSTED ; } + +mark_reg_known_zero (env, regs,BPF_REG_0 ); + regs[BPF_REG_0 ].btf = desc_btf; + regs[BPF_REG_0 ].type = type; + regs[BPF_REG_0 ].btf_id = ptr_type_id; }if (is_kfunc_ret_null (&meta)) {
tools/testing/selftests/bpf/progs/map_kptr_fail.c +2 -2
@@ -272,7 +272,7 @@ int reject_untrusted_xchg(struct __sk_buff *ctx) SEC("?tc") __failure -__msg("invalid kptr access, R2 type=ptr_prog_test_ref_kfunc expected=ptr_prog_test_member") +__msg("invalid kptr access, R2 type=trusted_ptr_prog_test_ref_kfunc expected=ptr_prog_test_member")int reject_bad_type_xchg (struct __sk_buff *ctx) {struct prog_test_ref_kfunc *ref_ptr;
@@ -291,7 +291,7 @@ int reject_bad_type_xchg(struct __sk_buff *ctx) } SEC("?tc") -__failure __msg("invalid kptr access, R2 type=ptr_prog_test_ref_kfunc") +__failure __msg("invalid kptr access, R2 type=trusted_ptr_prog_test_ref_kfunc") int reject_member_of_ref_xchg(struct __sk_buff *ctx) {struct prog_test_ref_kfunc *ref_ptr;
tools/testing/selftests/bpf/progs/struct_ops_kptr_return_fail__wrong_type.c +1 -1
@@ -12,7 +12,7 @@ void bpf_task_release(struct task_struct *p) __ksym;* reject programs returning a referenced kptr of the wrong type. */ SEC("struct_ops/test_return_ref_kptr") -__failure __msg("At program exit the register R0 is not a known value (ptr_or_null_)") +__failure __msg("At program exit the register R0 is not a known value (trusted_ptr_or_null_)")struct task_struct *BPF_PROG (kptr_return_fail__wrong_type ,int dummy,struct task_struct *task,struct cgroup *cgrp) {
tools/testing/selftests/bpf/progs/verifier_global_ptr_args.c +1 -1
@@ -72,7 +72,7 @@ int trusted_task_arg_nonnull_fail1(void *ctx) SEC("?tp_btf/task_newtask") __failure __log_level(2) -__msg("R1 type=ptr_or_null_ expected=ptr_, trusted_ptr_, rcu_ptr_") +__msg("R1 type=trusted_ptr_or_null_ expected=ptr_, trusted_ptr_, rcu_ptr_") __msg("Caller passes invalid args into func#1 ('subprog_trusted_task_nonnull')")int trusted_task_arg_nonnull_fail2 (void *ctx) {
tools/testing/selftests/bpf/verifier/calls.c +1 -1
@@ -220,7 +220,7 @@ }, .result_unpriv = REJECT, .result = REJECT, - .errstr = "variable ptr_ access var_off=(0x0; 0x7) disallowed", + .errstr = "variable trusted_ptr_ access var_off=(0x0; 0x7) disallowed", }, { "calls: invalid kfunc call: referenced arg needs refcounted PTR_TO_BTF_ID",
Signature · 1 line
2.52.0.457.g6b5491de43-goog
PATCH 2/3 bpf: drop KF_ACQUIRE flag on BPF kfunc bpf_get_root_mem_cgroup()
With the BPF verifier now treating pointers to struct types returned from BPF kfuncs as implicitly trusted by default, there is no need for bpf_get_root_mem_cgroup() to be annotated with the KF_ACQUIRE flag.
bpf_get_root_mem_cgroup() does not acquire any references, but rather simply returns a NULL pointer or a pointer to a struct mem_cgroup object that is valid for the entire lifetime of the kernel.
This simplifies BPF programs using this kfunc by removing the requirement to pair the call with bpf_put_mem_cgroup().
--- mm/bpf_memcontrol.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-)
mm/bpf_memcontrol.c +7 -7
@@ -13,12 +13,12 @@ __bpf_kfunc_start_defs();/** * bpf_get_root_mem_cgroup - Returns a pointer to the root memory cgroup * -* The function has KF_ACQUIRE semantics, even though the root memory -* cgroup is never destroyed after being created and doesn't require -* reference counting. And it's perfectly safe to pass it to -* bpf_put_mem_cgroup() -* -* Return: A pointer to the root memory cgroup. +* Return: A pointer to the root memory cgroup. Notably, the pointer to the +* returned struct mem_cgroup is trusted by default, so it's perfectably +* acceptable to also pass this pointer into other BPF kfuncs (e.g., +* bpf_mem_cgroup_usage()). Additionally, this BPF kfunc does not make use of +* KF_ACQUIRE semantics, so there's no requirement for the BPF program to call +* bpf_put_mem_cgroup() on the pointer returned by this BPF kfunc. */ __bpf_kfunc struct mem_cgroup *bpf_get_root_mem_cgroup(void) {
@@ -162,7 +162,7 @@ __bpf_kfunc void bpf_mem_cgroup_flush_stats(struct mem_cgroup *memcg)__bpf_kfunc_end_defs (); BTF_KFUNCS_START(bpf_memcontrol_kfuncs) -BTF_ID_FLAGS(func, bpf_get_root_mem_cgroup, KF_ACQUIRE | KF_RET_NULL) +BTF_ID_FLAGS(func, bpf_get_root_mem_cgroup, KF_RET_NULL) BTF_ID_FLAGS(func, bpf_get_mem_cgroup, KF_ACQUIRE | KF_RET_NULL | KF_RCU) BTF_ID_FLAGS(func, bpf_put_mem_cgroup, KF_RELEASE)
Signature · 1 line
2.52.0.457.g6b5491de43-goog
Re: [PATCH bpf-next 2/3] bpf: drop KF_ACQUIRE flag on BPF kfunc bpf_get_root_mem_cgroup()
Quoted 9 lines · Matt Bobrowski
With the BPF verifier now treating pointers to struct types returned from BPF kfuncs as implicitly trusted by default, there is no need for
bpf_get_root_mem_cgroup()to be annotated with theKF_ACQUIREflag.
bpf_get_root_mem_cgroup()does not acquire any references, but rather simply returns a NULL pointer or a pointer to a structmem_cgroupobject that is valid for the entire lifetime of the kernel.
Quoted 5 lines
This simplifies BPF programs using this kfunc by removing the requirement to pair the call with
bpf_put_mem_cgroup().Signed-off-by Matt Bobrowski <[email protected]>---
Roman Gushchin
maintainerRe: Subject: [PATCH bpf-next 2/3] bpf: drop KF_ACQUIRE flag on BPF kfunc bpf_get_root_mem_cgroup()
In reply to Matt BobrowskiQuoted 11 lines
With the BPF verifier now treating pointers to struct types returned from BPF kfuncs as implicitly trusted by default, there is no need for
bpf_get_root_mem_cgroup()to be annotated with theKF_ACQUIREflag.
bpf_get_root_mem_cgroup()does not acquire any references, but rather simply returns a NULL pointer or a pointer to a structmem_cgroupobject that is valid for the entire lifetime of the kernel.This simplifies BPF programs using this kfunc by removing the requirement to pair the call with
bpf_put_mem_cgroup().
It's actually the opposite: having the get semantics (which is also suggested by the name) allows to treat the root memory cgroup exactly as any other. And it makes the code much simpler, otherwise you need to have these ugly checks across the codebase:
if (memcg != root_mem_cgroup) css_put(&memcg->css);
This is why __all__ memcg && cgroup code follows this principle and the hides the special handling of the root memory cgroup within css_get()/css_put().
I wasn't cc'ed on this series, otherwise I'd nack this patch. If the overhead of an extra kfunc call is a concern here (which I doubt), we can introduce a non-acquire bpf_root_mem_cgroup() version.
And I strongly suggest to revert this change.
Thanks
Re: Subject: [PATCH bpf-next 2/3] bpf: drop KF_ACQUIRE flag on BPF kfunc bpf_get_root_mem_cgroup()
Quoted 12 lines · Roman Gushchin
Quoted 11 lines
With the BPF verifier now treating pointers to struct types returned from BPF kfuncs as implicitly trusted by default, there is no need for
bpf_get_root_mem_cgroup()to be annotated with theKF_ACQUIREflag.
bpf_get_root_mem_cgroup()does not acquire any references, but rather simply returns a NULL pointer or a pointer to a structmem_cgroupobject that is valid for the entire lifetime of the kernel.This simplifies BPF programs using this kfunc by removing the requirement to pair the call with
bpf_put_mem_cgroup().
Quoted 6 lines
It's actually the opposite: having the get semantics (which is also suggested by the name) allows to treat the root memory cgroup exactly as any other. And it makes the code much simpler, otherwise you need to have these ugly checks across the codebase:
if (memcg != root_mem_cgroup) css_put(&memcg->css);
I mean, you're certainly not forced to do this. But, I do also see what you mean.
Quoted 10 lines
This is why
__all__memcg && cgroup code follows this principle and the hides the special handling of the root memory cgroup within css_get()/css_put().I wasn't cc'ed on this series, otherwise I'd nack this patch. If the overhead of an extra kfunc call is a concern here (which I doubt), we can introduce a non-acquire
bpf_root_mem_cgroup()version.And I strongly suggest to revert this change.
Apologies, I honestly thought I did CC you on this series. Don't know what happened with that. Anyway, I'm totally OK with reverting this patch and keeping bpf_get_root_mem_cgroup() with KF_ACQUIRE semantics. bpf_get_root_mem_cgroup() was selected as it was the very first BPF kfunc that came to mind where implicit trusted pointer semantics should be applied by the BPF verifier.
Notably, the follow up selftest patch [0] will also need to be reverted if so as it relies on bpf_get_root_mem_cgroup() without KF_ACQUIRE. We can probably
[0] https://lore.kernel.org/bpf/[email protected]/T/#mfa14fb83b3350c25f961fd43dc4df9b25d00c5f5
Re: Subject: [PATCH bpf-next 2/3] bpf: drop KF_ACQUIRE flag on BPF kfunc bpf_get_root_mem_cgroup()
Quoted 42 lines · Matt Bobrowski
Quoted 12 lines · Roman Gushchin
Quoted 11 lines
With the BPF verifier now treating pointers to struct types returned from BPF kfuncs as implicitly trusted by default, there is no need for
bpf_get_root_mem_cgroup()to be annotated with theKF_ACQUIREflag.
bpf_get_root_mem_cgroup()does not acquire any references, but rather simply returns a NULL pointer or a pointer to a structmem_cgroupobject that is valid for the entire lifetime of the kernel.This simplifies BPF programs using this kfunc by removing the requirement to pair the call with
bpf_put_mem_cgroup().Quoted 6 lines
It's actually the opposite: having the get semantics (which is also suggested by the name) allows to treat the root memory cgroup exactly as any other. And it makes the code much simpler, otherwise you need to have these ugly checks across the codebase:
if (memcg != root_mem_cgroup) css_put(&memcg->css);I mean, you're certainly not forced to do this. But, I do also see what you mean.
Quoted 10 lines
This is why
__all__memcg && cgroup code follows this principle and the hides the special handling of the root memory cgroup within css_get()/css_put().I wasn't cc'ed on this series, otherwise I'd nack this patch. If the overhead of an extra kfunc call is a concern here (which I doubt), we can introduce a non-acquire
bpf_root_mem_cgroup()version.And I strongly suggest to revert this change.
Apologies, I honestly thought I did CC you on this series. Don't know what happened with that. Anyway, I'm totally OK with reverting this patch and keeping
bpf_get_root_mem_cgroup()withKF_ACQUIREsemantics.bpf_get_root_mem_cgroup()was selected as it was the very first BPF kfunc that came to mind where implicit trusted pointer semantics should be applied by the BPF verifier.
Quoted 5 lines
Notably, the follow up selftest patch [0] will also need to be reverted if so as it relies on
bpf_get_root_mem_cgroup()withoutKF_ACQUIRE. We can probably[0] https://lore.kernel.org/bpf/[email protected]/T/#mfa14fb83b3350c25f961fd43dc4df9b25d00c5f5
Instead of revert of two patches, let's revert one and replace with test kfunc that 2nd patch can use.
tbh I don't think it's a big deal in practice.
Kernel code working with cgroups might be different than bpf. I'm not sure what was the use case for bpf_get_root_mem_cgroup().
Roman,
please share your protype bpf code for oom, so it's easier to see why non-acquire semantics for bpf_get_root_mem_cgroup() are problematic.
Re: Subject: [PATCH bpf-next 2/3] bpf: drop KF_ACQUIRE flag on BPF kfunc bpf_get_root_mem_cgroup()
Quoted 54 lines · Alexei Starovoitov
Quoted 42 lines · Matt Bobrowski
Quoted 12 lines · Roman Gushchin
Quoted 11 lines
With the BPF verifier now treating pointers to struct types returned from BPF kfuncs as implicitly trusted by default, there is no need for
bpf_get_root_mem_cgroup()to be annotated with theKF_ACQUIREflag.
bpf_get_root_mem_cgroup()does not acquire any references, but rather simply returns a NULL pointer or a pointer to a structmem_cgroupobject that is valid for the entire lifetime of the kernel.This simplifies BPF programs using this kfunc by removing the requirement to pair the call with
bpf_put_mem_cgroup().Quoted 6 lines
It's actually the opposite: having the get semantics (which is also suggested by the name) allows to treat the root memory cgroup exactly as any other. And it makes the code much simpler, otherwise you need to have these ugly checks across the codebase:
if (memcg != root_mem_cgroup) css_put(&memcg->css);I mean, you're certainly not forced to do this. But, I do also see what you mean.
Quoted 10 lines
This is why
__all__memcg && cgroup code follows this principle and the hides the special handling of the root memory cgroup within css_get()/css_put().I wasn't cc'ed on this series, otherwise I'd nack this patch. If the overhead of an extra kfunc call is a concern here (which I doubt), we can introduce a non-acquire
bpf_root_mem_cgroup()version.And I strongly suggest to revert this change.
Apologies, I honestly thought I did CC you on this series. Don't know what happened with that. Anyway, I'm totally OK with reverting this patch and keeping
bpf_get_root_mem_cgroup()withKF_ACQUIREsemantics.bpf_get_root_mem_cgroup()was selected as it was the very first BPF kfunc that came to mind where implicit trusted pointer semantics should be applied by the BPF verifier.Quoted 5 lines
Notably, the follow up selftest patch [0] will also need to be reverted if so as it relies on
bpf_get_root_mem_cgroup()withoutKF_ACQUIRE. We can probably[0] https://lore.kernel.org/bpf/[email protected]/T/#mfa14fb83b3350c25f961fd43dc4df9b25d00c5f5
Instead of revert of two patches, let's revert one and replace with test kfunc that 2nd patch can use.
Quoted 7 lines
tbh I don't think it's a big deal in practice.
Kernel code working with cgroups might be different than bpf. I'm not sure what was the use case forbpf_get_root_mem_cgroup().Roman,
please share your protype bpf code for oom, so it's easier to see why non-acquire semantics forbpf_get_root_mem_cgroup()are problematic.
Actually, thinking more about it, bpf_get_root_mem_cgroup() should NOT have an acquire semantics, otherwise you cannot even implement:
static inline bool mem_cgroup_is_root(struct mem_cgroup *memcg)
{
return (memcg == root_mem_cgroup);
}without ugliness:
static inline bool bpf_mem_cgroup_is_root(struct mem_cgroup *memcg)
{
struct mem_cgroup *root_memcg = bpf_get_root_mem_cgroup();
bool ret = memcg == root_memcg;
bpf_put_mem_cgroup(root_memcg);
return ret;
}If pattern memcg != root_mem_cgroup happens often above 'put' overhead will hurt performance.
More thoughts...
maybe we should teach the verifier to expose struct mem_cgroup *root_mem_cgroup;
as a trusted pointer and remove this kfunc.
Also gate the whole feature by CONFIG_MEMCG.
Check for != NULL after every memcg_get will cause ugly code and runtime penalty for a rare case of !config_memcg.
Roman Gushchin
maintainerRe: Subject: [PATCH bpf-next 2/3] bpf: drop KF_ACQUIRE flag on BPF kfunc bpf_get_root_mem_cgroup()
Quoted 64 lines · Alexei Starovoitov
Quoted 54 lines · Alexei Starovoitov
Quoted 42 lines · Matt Bobrowski
Quoted 12 lines · Roman Gushchin
Quoted 11 lines
With the BPF verifier now treating pointers to struct types returned from BPF kfuncs as implicitly trusted by default, there is no need for
bpf_get_root_mem_cgroup()to be annotated with theKF_ACQUIREflag.
bpf_get_root_mem_cgroup()does not acquire any references, but rather simply returns a NULL pointer or a pointer to a structmem_cgroupobject that is valid for the entire lifetime of the kernel.This simplifies BPF programs using this kfunc by removing the requirement to pair the call with
bpf_put_mem_cgroup().Quoted 6 lines
It's actually the opposite: having the get semantics (which is also suggested by the name) allows to treat the root memory cgroup exactly as any other. And it makes the code much simpler, otherwise you need to have these ugly checks across the codebase:
if (memcg != root_mem_cgroup) css_put(&memcg->css);I mean, you're certainly not forced to do this. But, I do also see what you mean.
Quoted 10 lines
This is why
__all__memcg && cgroup code follows this principle and the hides the special handling of the root memory cgroup within css_get()/css_put().I wasn't cc'ed on this series, otherwise I'd nack this patch. If the overhead of an extra kfunc call is a concern here (which I doubt), we can introduce a non-acquire
bpf_root_mem_cgroup()version.And I strongly suggest to revert this change.
Apologies, I honestly thought I did CC you on this series. Don't know what happened with that. Anyway, I'm totally OK with reverting this patch and keeping
bpf_get_root_mem_cgroup()withKF_ACQUIREsemantics.bpf_get_root_mem_cgroup()was selected as it was the very first BPF kfunc that came to mind where implicit trusted pointer semantics should be applied by the BPF verifier.Quoted 5 lines
Notably, the follow up selftest patch [0] will also need to be reverted if so as it relies on
bpf_get_root_mem_cgroup()withoutKF_ACQUIRE. We can probably[0] https://lore.kernel.org/bpf/[email protected]/T/#mfa14fb83b3350c25f961fd43dc4df9b25d00c5f5
Instead of revert of two patches, let's revert one and replace with test kfunc that 2nd patch can use.
Quoted 7 lines
tbh I don't think it's a big deal in practice.
Kernel code working with cgroups might be different than bpf. I'm not sure what was the use case forbpf_get_root_mem_cgroup().Roman,
please share your protype bpf code for oom, so it's easier to see why non-acquire semantics forbpf_get_root_mem_cgroup()are problematic.
Quoted 7 lines
Actually, thinking more about it,
bpf_get_root_mem_cgroup()should NOT have an acquire semantics, otherwise you cannot even implement:static inline bool
mem_cgroup_is_root(struct mem_cgroup *memcg){ return (memcg == root_mem_cgroup); }
You can check memcg->css.parent == NULL instead.
Quoted 11 lines
without ugliness:
static inline bool
bpf_mem_cgroup_is_root(struct mem_cgroup *memcg){ struct mem_cgroup *root_memcg = bpf_get_root_mem_cgroup(); bool ret = memcg == root_memcg; bpf_put_mem_cgroup(root_memcg); return ret; }
Maybe we need both, but if root_mem_cgroup is handled different, you can't do a very natural thing like:
some_func (struct *mem_cgroup subtree_root) {
struct mem_cgroup *memcg = subtree_root ? subtree_root : bpf_get_root_mem_cgroup();
// iterate over subtreeor you can't pass a pointer (with a reference) to a function or a work with the assumption that it should drop the reference at the end.
Basically you can't easily mix the root_mem_cgroup pointer with normal memcg pointers.
E.g. in my bpfoom case:
SEC("struct_ops.s/handle_out_of_memory")
int BPF_PROG(test_out_of_memory, struct oom_control *oc, struct bpf_struct_ops_link *link)
{
struct task_struct *task;
struct mem_cgroup *root_memcg = oc->memcg;
struct mem_cgroup *memcg, *victim = NULL;
struct cgroup_subsys_state *css_pos, *css;
unsigned long usage, max_usage = 0;
unsigned long pagecache = 0;
int ret = 0;
if (root_memcg)
root_memcg = bpf_get_mem_cgroup(&root_memcg->css);
else
root_memcg = bpf_get_root_mem_cgroup();
if (!root_memcg)
return 0;
css = &root_memcg->css;
if (css && css->cgroup == link->cgroup)
goto exit;
bpf_rcu_read_lock();
bpf_for_each(css, css_pos, &root_memcg->css, BPF_CGROUP_ITER_DESCENDANTS_POST) {
if (css_pos->cgroup->nr_descendants + css_pos->cgroup->nr_dying_descendants)
continue;
memcg = bpf_get_mem_cgroup(css_pos);
if (!memcg)
continue;
< ... >
bpf_put_mem_cgroup(memcg);
}
bpf_rcu_read_unlock();
< ... >
bpf_put_mem_cgroup(victim);
exit:
bpf_put_mem_cgroup(root_memcg);
return ret;
}--
How to write it without get semantics?
Thanks!
Re: Subject: [PATCH bpf-next 2/3] bpf: drop KF_ACQUIRE flag on BPF kfunc bpf_get_root_mem_cgroup()
Quoted 9 lines · Roman Gushchin
E.g. in my bpfoom case:
SEC("struct_ops.s/handle_out_of_memory")
intBPF_PROG(test_out_of_memory, struct oom_control *oc, struct bpf_struct_ops_link *link){ struct task_struct *task; struct mem_cgroup *root_memcg = oc->memcg;
And you'll annotate oom_control->memcg with BTF_TYPE_SAFE_TRUSTED_OR_NULL ?
Quoted 8 lines
struct mem_cgroup *memcg, *victim = NULL;
struct cgroup_subsys_state *css_pos, *css;
unsigned long usage, max_usage = 0;
unsigned long pagecache = 0;
int ret = 0;
if (root_memcg)
root_memcg = bpf_get_mem_cgroup(&root_memcg->css);similar for mem_cgroup and css types ?
or as BTF_TYPE_SAFE_RCU_OR_NULL ?
Quoted 11 lines
else
root_memcg = bpf_get_root_mem_cgroup();
if (!root_memcg)
return 0;
css = &root_memcg->css;
if (css && css->cgroup == link->cgroup)
goto exit;
bpf_rcu_read_lock();then this is a bug ? and rcu_read_lock needs to move up?
Quoted 11 lines
bpf_for_each(css, css_pos, &root_memcg->css, BPF_CGROUP_ITER_DESCENDANTS_POST) {
if (css_pos->cgroup->nr_descendants + css_pos->cgroup->nr_dying_descendants)
continue;
memcg = bpf_get_mem_cgroup(css_pos);
if (!memcg)
continue;
< ... >
bpf_put_mem_cgroup(memcg);Quoted 8 lines
}
bpf_rcu_read_unlock();
< ... >
bpf_put_mem_cgroup(victim);
exit:
bpf_put_mem_cgroup(root_memcg);Fair enough.
Looks like quite a few pieces are still missing for this to work end-to-end, but, sure, let's revert back to acquire semantics.
Matt,
please come with a way to fix a selftest. Introduce test kfunc or something.
Re: Subject: [PATCH bpf-next 2/3] bpf: drop KF_ACQUIRE flag on BPF kfunc bpf_get_root_mem_cgroup()
Quoted 59 lines · Alexei Starovoitov
Quoted 9 lines · Roman Gushchin
E.g. in my bpfoom case:
SEC("struct_ops.s/handle_out_of_memory")
intBPF_PROG(test_out_of_memory, struct oom_control *oc, struct bpf_struct_ops_link *link){ struct task_struct *task; struct mem_cgroup *root_memcg = oc->memcg;And you'll annotate
oom_control->memcgwithBTF_TYPE_SAFE_TRUSTED_OR_NULL?Quoted 8 lines
struct mem_cgroup *memcg, *victim = NULL; struct cgroup_subsys_state *css_pos, *css; unsigned long usage, max_usage = 0; unsigned long pagecache = 0; int ret = 0; if (root_memcg) root_memcg = bpf_get_mem_cgroup(&root_memcg->css);similar for
mem_cgroupand css types ?
or asBTF_TYPE_SAFE_RCU_OR_NULL?Quoted 11 lines
else root_memcg = bpf_get_root_mem_cgroup(); if (!root_memcg) return 0; css = &root_memcg->css; if (css && css->cgroup == link->cgroup) goto exit; bpf_rcu_read_lock();then this is a bug ? and
rcu_read_lockneeds to move up?Quoted 11 lines
bpf_for_each(css, css_pos, &root_memcg->css, BPF_CGROUP_ITER_DESCENDANTS_POST) { if (css_pos->cgroup->nr_descendants + css_pos->cgroup->nr_dying_descendants) continue; memcg = bpf_get_mem_cgroup(css_pos); if (!memcg) continue; < ... > bpf_put_mem_cgroup(memcg);Quoted 8 lines
} bpf_rcu_read_unlock(); < ... > bpf_put_mem_cgroup(victim); exit: bpf_put_mem_cgroup(root_memcg);
Quoted 6 lines
Fair enough.
Looks like quite a few pieces are still missing for this to work end-to-end, but, sure, let's revert back to acquire semantics.Matt,
please come with a way to fix a selftest. Introduce test kfunc or something.
Already on it.
Re: Subject: [PATCH bpf-next 2/3] bpf: drop KF_ACQUIRE flag on BPF kfunc bpf_get_root_mem_cgroup()
Quoted 60 lines · Matt Bobrowski
Quoted 59 lines · Alexei Starovoitov
Quoted 9 lines · Roman Gushchin
E.g. in my bpfoom case:
SEC("struct_ops.s/handle_out_of_memory")
intBPF_PROG(test_out_of_memory, struct oom_control *oc, struct bpf_struct_ops_link *link){ struct task_struct *task; struct mem_cgroup *root_memcg = oc->memcg;And you'll annotate
oom_control->memcgwithBTF_TYPE_SAFE_TRUSTED_OR_NULL?Quoted 8 lines
struct mem_cgroup *memcg, *victim = NULL; struct cgroup_subsys_state *css_pos, *css; unsigned long usage, max_usage = 0; unsigned long pagecache = 0; int ret = 0; if (root_memcg) root_memcg = bpf_get_mem_cgroup(&root_memcg->css);similar for
mem_cgroupand css types ?
or asBTF_TYPE_SAFE_RCU_OR_NULL?Quoted 11 lines
else root_memcg = bpf_get_root_mem_cgroup(); if (!root_memcg) return 0; css = &root_memcg->css; if (css && css->cgroup == link->cgroup) goto exit; bpf_rcu_read_lock();then this is a bug ? and
rcu_read_lockneeds to move up?Quoted 19 lines
bpf_for_each(css, css_pos, &root_memcg->css, BPF_CGROUP_ITER_DESCENDANTS_POST) { if (css_pos->cgroup->nr_descendants + css_pos->cgroup->nr_dying_descendants) continue; memcg = bpf_get_mem_cgroup(css_pos); if (!memcg) continue; < ... > bpf_put_mem_cgroup(memcg); } bpf_rcu_read_unlock(); < ... > bpf_put_mem_cgroup(victim); exit: bpf_put_mem_cgroup(root_memcg);
Quoted 8 lines
Quoted 6 lines
Fair enough.
Looks like quite a few pieces are still missing for this to work end-to-end, but, sure, let's revert back to acquire semantics.Matt,
please come with a way to fix a selftest. Introduce test kfunc or something.Already on it.
Here is the updated selftest which relies on new test specific BPF kfuncs defined within bpf_testmod instead:
Roman Gushchin
maintainerRe: Subject: [PATCH bpf-next 2/3] bpf: drop KF_ACQUIRE flag on BPF kfunc bpf_get_root_mem_cgroup()
In reply to Alexei StarovoitovQuoted 5 lines · Alexei Starovoitov
Quoted 4 lines · Roman Gushchin
E.g. in my bpfoom case:
Quoted 8 lines
Quoted 5 lines
SEC("struct_ops.s/handle_out_of_memory")
intBPF_PROG(test_out_of_memory, struct oom_control *oc, struct bpf_struct_ops_link *link){ struct task_struct *task; struct mem_cgroup *root_memcg = oc->memcg;And you'll annotate
oom_control->memcgwithBTF_TYPE_SAFE_TRUSTED_OR_NULL?
Yes.
Quoted 7 lines
Quoted 7 lines
struct mem_cgroup *memcg, *victim = NULL; struct cgroup_subsys_state *css_pos, *css; unsigned long usage, max_usage = 0; unsigned long pagecache = 0; int ret = 0;
Quoted 5 lines
Quoted 2 lines
if (root_memcg) root_memcg = bpf_get_mem_cgroup(&root_memcg->css);similar for
mem_cgroupand css types ?
or asBTF_TYPE_SAFE_RCU_OR_NULL?
css is embedded into memcg, so in theory it doesn't require it. also bpf_get_mem_cgroup() has acquire semantics, so in my understanding, it can take non-trusted arguments.
Quoted 7 lines
Quoted 7 lines
else root_memcg = bpf_get_root_mem_cgroup(); if (!root_memcg) return 0;
Quoted 7 lines
Quoted 5 lines
css = &root_memcg->css; if (css && css->cgroup == link->cgroup) goto exit; bpf_rcu_read_lock();then this is a bug ? and
rcu_read_lockneeds to move up?
No, rcu read lock is required to protect the iterator within the for each loop. root_memcg is protected by a bumped ref counter.
Quoted 17 lines
Quoted 17 lines
bpf_for_each(css, css_pos, &root_memcg->css, BPF_CGROUP_ITER_DESCENDANTS_POST) { if (css_pos->cgroup->nr_descendants + css_pos->cgroup->nr_dying_descendants) continue; memcg = bpf_get_mem_cgroup(css_pos); if (!memcg) continue; < ... > bpf_put_mem_cgroup(memcg); } bpf_rcu_read_unlock(); < ... >
Quoted 7 lines
Quoted 3 lines
bpf_put_mem_cgroup(victim); exit: bpf_put_mem_cgroup(root_memcg);Fair enough.
Looks like quite a few pieces are still missing for this to work end-to-end, but, sure, let's revert back to acquire semantics.
It's close, I hope to send out v3 in few days.
I was rebasing to the latest bpf-next and then noticed that the test is not working anymore because of this change.
Quoted 4 lines
Matt,
please come with a way to fix a selftest. Introduce test kfunc or something.
Thanks!
Re: Subject: [PATCH bpf-next 2/3] bpf: drop KF_ACQUIRE flag on BPF kfunc bpf_get_root_mem_cgroup()
Quoted 6 lines · Roman Gushchin
Quoted 6 lines
Matt,
please come with a way to fix a selftest. Introduce test kfunc or something.
Applied the tweak for selftest.
Matt,
please send a patch to go back to acquire behavior and summarize this thread in the commit log.
Re: Subject: [PATCH bpf-next 2/3] bpf: drop KF_ACQUIRE flag on BPF kfunc bpf_get_root_mem_cgroup()
Quoted 9 lines · Alexei Starovoitov
Quoted 6 lines · Roman Gushchin
Quoted 6 lines
Matt,
please come with a way to fix a selftest. Introduce test kfunc or something.Applied the tweak for selftest.
Thank you Alexei!
Quoted 3 lines
Matt,
please send a patch to go back to acquire behavior and summarize this thread in the commit log.
Sure, and here it is:
- https://lore.kernel.org/bpf/[email protected]/T/#u
PATCH 3/3 selftests/bpf: assert BPF kfunc default trusted pointer semantics
In reply to Matt BobrowskiThe BPF verifier was recently updated to treat pointers to struct types returned from BPF kfuncs as implicitly trusted by default. Add a new test case to exercise this new implicit trust semantic.
The KF_ACQUIRE flag was dropped from the bpf_get_root_mem_cgroup() kfunc because it returns a global pointer to root_mem_cgroup without performing any explicit reference counting. This makes it an ideal candidate to verify the new implicit trusted pointer semantics.
--- .../selftests/bpf/prog_tests/verifier.c | 2 ++ .../selftests/bpf/progs/verifier_memcontrol.c | 32 +++++++++++++++++++ 2 files changed, 34 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/verifier_memcontrol.c
tools/testing/selftests/bpf/prog_tests/verifier.c +2 -0
@@ -61,6 +61,7 @@#include "verifier_masking.skel.h" #include "verifier_may_goto_1.skel.h" #include "verifier_may_goto_2.skel.h" +#include "verifier_memcontrol.skel.h" #include "verifier_meta_access.skel.h" #include "verifier_movsx.skel.h" #include "verifier_mtu.skel.h"
@@ -202,6 +203,7 @@ void test_verifier_map_ret_val(void) { RUN(verifier_map_ret_val); }void test_verifier_masking (void ) {RUN (verifier_masking); }void test_verifier_may_goto_1 (void ) {RUN (verifier_may_goto_1); }void test_verifier_may_goto_2 (void ) {RUN (verifier_may_goto_2); } +void test_verifier_memcontrol (void ) {RUN (verifier_memcontrol); }void test_verifier_meta_access (void ) {RUN (verifier_meta_access); }void test_verifier_movsx (void ) {RUN (verifier_movsx); }void test_verifier_mul (void ) {RUN (verifier_mul); }
tools/testing/selftests/bpf/progs/verifier_memcontrol.c new file +32 -0
@@ -0,0 +1,32 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Copyright 2026 Google LLC. +*/ + +#include <vmlinux.h> +#include <bpf/bpf_helpers.h> +#include <bpf/bpf_tracing.h> +#include "bpf_misc.h" + +SEC("syscall") +__success __retval(0) +int root_mem_cgroup_default_trusted(void *ctx) +{ +unsigned long usage; +struct mem_cgroup *root_mem_cgroup; + + root_mem_cgroup =bpf_get_root_mem_cgroup (); +if (!root_mem_cgroup) +return 1 ; + +/* +* BPF kfunc bpf_get_root_mem_cgroup() returns a PTR_TO_BTF_ID | +* PTR_TRUSTED | PTR_MAYBE_NULL, therefore it should be accepted when +* passed to a BPF kfunc only accepting KF_TRUSTED_ARGS. +*/ + usage =bpf_mem_cgroup_usage (root_mem_cgroup); +__sink (usage); +return 0 ; +} + +char _license[] SEC("license") = "GPL";
Signature · 1 line
2.52.0.457.g6b5491de43-goog
Re: [PATCH bpf-next 3/3] selftests/bpf: assert BPF kfunc default trusted pointer semantics
Quoted 5 lines · Matt Bobrowski
The BPF verifier was recently updated to treat pointers to struct types returned from BPF kfuncs as implicitly trusted by default. Add a new test case to exercise this new implicit trust semantic.
Quoted 7 lines
The
KF_ACQUIREflag was dropped from thebpf_get_root_mem_cgroup()kfunc because it returns a global pointer toroot_mem_cgroupwithout performing any explicit reference counting. This makes it an ideal candidate to verify the new implicit trusted pointer semantics.Signed-off-by Matt Bobrowski <[email protected]>---
Quoted 6 lines · Matt Bobrowski
Teach the BPF verifier to treat pointers to struct types returned from BPF kfuncs as implicitly trusted (
PTR_TO_BTF_ID|PTR_TRUSTED) by default. Returning untrusted pointers to struct types from BPF kfuncs should be considered an exception only, and certainly not the norm.
Quoted 7 lines
Update existing selftests to reflect the change in register type printing (e.g.
ptr_becomingtrusted_ptr_in verifier error messages).Signed-off-by Matt Bobrowski <[email protected]>---
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Hello:
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Quoted 10 lines · On Tue, 13 Jan 2026 08:39:47 +0000 you wrote:
Teach the BPF verifier to treat pointers to struct types returned from BPF kfuncs as implicitly trusted (
PTR_TO_BTF_ID|PTR_TRUSTED) by default. Returning untrusted pointers to struct types from BPF kfuncs should be considered an exception only, and certainly not the norm.Update existing selftests to reflect the change in register type printing (e.g.
ptr_becomingtrusted_ptr_in verifier error messages).[...]
Here is the summary with links:
- [bpf-next,1/3] bpf: return PTR_TO_BTF_ID | PTR_TRUSTED from BPF kfuncs by default
https://git.kernel.org/bpf/bpf-next/c/f8ade2342e22 - [bpf-next,2/3] bpf: drop KF_ACQUIRE flag on BPF kfunc bpf_get_root_mem_cgroup() https://git.kernel.org/bpf/bpf-next/c/e463b6de9da1 - [bpf-next,3/3] selftests/bpf: assert BPF kfunc default trusted pointer semantics https://git.kernel.org/bpf/bpf-next/c/bbdbed193bcf
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html