bpf: Warn with bpf_unreachable() kfunc maybe due to uninitialized variable
Subsystems (4)
Yonghong Song
reviewerv4 cover bpf: Warn with bpf_unreachable() kfunc maybe due to uninitialized variable
Marc Suñé (Isovalent, part of Cisco) reported an issue where an uninitialized variable caused generating bpf prog binary code not working as expected. The reproducer is in [1] where the flags “-Wall -Werror” are enabled, but there is no warning as the compiler takes advantage of uninitialized variable to do aggressive optimization. Such optimization results in a verification log:
last insn is not an exit or jmp
User still needs to take quite some time to figure out what is the root cause.
To give a better hint to user, bpf_unreachable() kfunc is introduced in kernel and the compiler ([2]) will encode bpf_unreachable() as needed. For example, compiler may generate 'unreachable' IR after do optimizaiton by taking advantage of uninitialized variable, and later bpf backend will translate such 'unreachable' IR to bpf_unreachable() func in final binary. When kernel detects bpf_unreachable(), it is able to issue much better verifier log, e.g.
unexpected bpf_unreachable() due to uninitialized variable?
[1] https://github.com/msune/clang_bpf/blob/main/Makefile#L3
[2] https://github.com/llvm/llvm-project/pull/131731
Changelogs:
v3 -> v4:
- v3: https://lore.kernel.org/bpf/[email protected]/
- Remove special_kfunc_set in verifier.
v2 -> v3:
- v2: https://lore.kernel.org/bpf/CAADnVQL9A8vB-yRjnZn8bgMrfDSO17FFBtS_xOs5w-LSq+p74g@mail.gmail.com/
- The newer llvm patch (above [2]) added 'exit' insn if the last insn
in the function is bpf_unreachable(). This way, check_subprogs()
handling is unnecessary and removed.
- Remove the big C test (above [1]) and add a simple C test and three
inline asm tests.
v1 -> v2:
- v1: https://lore.kernel.org/bpf/[email protected]/
- If bpf_unreachable() is hit during check_kfunc_call(), report the verification failure.
- Add three inline asm test cases.
Yonghong Song (3):
bpf: Remove special_kfunc_set from verifier
bpf: Warn with bpf_unreachable() kfunc maybe due to uninitialized
variable
selftests/bpf: Add unit tests with bpf_unreachable() kfunc
kernel/bpf/helpers.c | 5 + kernel/bpf/verifier.c | 381 +++++++++--------- .../selftests/bpf/prog_tests/verifier.c | 2 + .../bpf/progs/verifier_bpf_unreachable.c | 61 +++ 4 files changed, 252 insertions(+), 197 deletions(-) create mode 100644 tools/testing/selftests/bpf/progs/verifier_bpf_unreachable.c
Signature · 1 line
2.47.1
Yonghong Song
reviewerPATCH v4 1/3 bpf: Remove special_kfunc_set from verifier
Currently, the verifier has both special_kfunc_set and special_kfunc_list. When adding a new kfunc usage to the verifier, it is often confusing about whether special_kfunc_set or special_kfunc_list or both should add that kfunc. For example, some kfuncs, e.g., bpf_dynptr_from_skb, bpf_dynptr_clone, bpf_wq_set_callback_impl, does not need to be in special_kfunc_set.
To avoid potential future confusion, special_kfunc_set is deleted
and btf_id_set_contains(&special_kfunc_set, ...) is removed.
The code is refactored with a new func check_special_kfunc(),
which contains all codes covered by original branch
meta.btf == btf_vmlinux && btf_id_set_contains(&special_kfunc_set, meta.func_id)
There is no functionality change.
--- kernel/bpf/verifier.c | 376 ++++++++++++++++++++---------------------- 1 file changed, 179 insertions(+), 197 deletions(-)
kernel/bpf/verifier.c +179 -197
@@ -12107,44 +12107,6 @@ enum special_kfunc_type { KF_bpf_res_spin_unlock_irqrestore, }; -BTF_SET_START(special_kfunc_set) -BTF_ID(func, bpf_obj_new_impl) -BTF_ID(func, bpf_obj_drop_impl) -BTF_ID(func, bpf_refcount_acquire_impl) -BTF_ID(func, bpf_list_push_front_impl) -BTF_ID(func, bpf_list_push_back_impl) -BTF_ID(func, bpf_list_pop_front) -BTF_ID(func, bpf_list_pop_back) -BTF_ID(func, bpf_list_front) -BTF_ID(func, bpf_list_back) -BTF_ID(func, bpf_cast_to_kern_ctx) -BTF_ID(func, bpf_rdonly_cast) -BTF_ID(func, bpf_rbtree_remove) -BTF_ID(func, bpf_rbtree_add_impl) -BTF_ID(func, bpf_rbtree_first) -BTF_ID(func, bpf_rbtree_root) -BTF_ID(func, bpf_rbtree_left) -BTF_ID(func, bpf_rbtree_right) -#ifdef CONFIG_NET -BTF_ID(func, bpf_dynptr_from_skb) -BTF_ID(func, bpf_dynptr_from_xdp) -#endif -BTF_ID(func, bpf_dynptr_slice) -BTF_ID(func, bpf_dynptr_slice_rdwr) -BTF_ID(func, bpf_dynptr_clone) -BTF_ID(func, bpf_percpu_obj_new_impl) -BTF_ID(func, bpf_percpu_obj_drop_impl) -BTF_ID(func, bpf_throw) -BTF_ID(func, bpf_wq_set_callback_impl) -#ifdef CONFIG_CGROUPS -BTF_ID(func, bpf_iter_css_task_new) -#endif -#ifdef CONFIG_BPF_LSM -BTF_ID(func, bpf_set_dentry_xattr) -BTF_ID(func, bpf_remove_dentry_xattr) -#endif -BTF_SET_END(special_kfunc_set) - BTF_ID_LIST(special_kfunc_list) BTF_ID(func, bpf_obj_new_impl) BTF_ID(func, bpf_obj_drop_impl)
@@ -13452,6 +13414,175 @@ static int fetch_kfunc_meta(struct bpf_verifier_env *env,return 0 ; } +static int check_special_kfunc (struct bpf_verifier_env *env,struct bpf_kfunc_call_arg_meta *meta, +struct bpf_reg_state *regs,struct bpf_insn_aux_data *insn_aux, +const struct btf_type *ptr_type,struct btf *desc_btf,bool *done) +{ +const struct btf_type *ret_t; +int err =0 ; + +if (meta->btf != btf_vmlinux) { + *done = false; +return 0 ; + } + +if (meta->func_id == special_kfunc_list[KF_bpf_obj_new_impl] || + meta->func_id == special_kfunc_list[KF_bpf_percpu_obj_new_impl]) { +struct btf_struct_meta *struct_meta; +struct btf *ret_btf; +u32 ret_btf_id; + +if (meta->func_id == special_kfunc_list[KF_bpf_obj_new_impl] && !bpf_global_ma_set) +return -ENOMEM ; + +if (((u64 )(u32 )meta->arg_constant.value) != meta->arg_constant.value) { +verbose (env,"local type ID argument must be in range [0, U32_MAX]\n" ); +return -EINVAL ; + } + + ret_btf = env->prog->aux->btf; + ret_btf_id = meta->arg_constant.value; + +/* This may be NULL due to user not supplying a BTF */ +if (!ret_btf) { +verbose (env,"bpf_obj_new/bpf_percpu_obj_new requires prog BTF\n" ); +return -EINVAL ; + } + + ret_t =btf_type_by_id (ret_btf, ret_btf_id); +if (!ret_t || !__btf_type_is_struct (ret_t)) { +verbose (env,"bpf_obj_new/bpf_percpu_obj_new type ID argument must be of a struct\n" ); +return -EINVAL ; + } + +if (meta->func_id == special_kfunc_list[KF_bpf_percpu_obj_new_impl]) { +if (ret_t->size >BPF_GLOBAL_PERCPU_MA_MAX_SIZE ) { +verbose (env,"bpf_percpu_obj_new type size (%d) is greater than %d\n" , + ret_t->size,BPF_GLOBAL_PERCPU_MA_MAX_SIZE ); +return -EINVAL ; + } + +if (!bpf_global_percpu_ma_set) { +mutex_lock (&bpf_percpu_ma_lock); +if (!bpf_global_percpu_ma_set) { +/* Charge memory allocated with bpf_global_percpu_ma to +* root memcg. The obj_cgroup for root memcg is NULL. +*/ + err =bpf_mem_alloc_percpu_init (&bpf_global_percpu_ma,NULL ); +if (!err) + bpf_global_percpu_ma_set = true; + } +mutex_unlock (&bpf_percpu_ma_lock); +if (err) +return err; + } + +mutex_lock (&bpf_percpu_ma_lock); + err =bpf_mem_alloc_percpu_unit_init (&bpf_global_percpu_ma, ret_t->size); +mutex_unlock (&bpf_percpu_ma_lock); +if (err) +return err; + } + + struct_meta =btf_find_struct_meta (ret_btf, ret_btf_id); +if (meta->func_id == special_kfunc_list[KF_bpf_percpu_obj_new_impl]) { +if (!__btf_type_is_scalar_struct (env, ret_btf, ret_t,0 )) { +verbose (env,"bpf_percpu_obj_new type ID argument must be of a struct of scalars\n" ); +return -EINVAL ; + } + +if (struct_meta) { +verbose (env,"bpf_percpu_obj_new type ID argument must not contain special fields\n" ); +return -EINVAL ; + } + } + +mark_reg_known_zero (env, regs,BPF_REG_0 ); + regs[BPF_REG_0 ].type =PTR_TO_BTF_ID |MEM_ALLOC ; + regs[BPF_REG_0 ].btf = ret_btf; + regs[BPF_REG_0 ].btf_id = ret_btf_id; +if (meta->func_id == special_kfunc_list[KF_bpf_percpu_obj_new_impl]) + regs[BPF_REG_0 ].type |=MEM_PERCPU ; + + insn_aux->obj_new_size = ret_t->size; + insn_aux->kptr_struct_meta = struct_meta; + }else if (meta->func_id == special_kfunc_list[KF_bpf_refcount_acquire_impl]) { +mark_reg_known_zero (env, regs,BPF_REG_0 ); + regs[BPF_REG_0 ].type =PTR_TO_BTF_ID |MEM_ALLOC ; + regs[BPF_REG_0 ].btf = meta->arg_btf; + regs[BPF_REG_0 ].btf_id = meta->arg_btf_id; + + insn_aux->kptr_struct_meta = +btf_find_struct_meta (meta->arg_btf, + meta->arg_btf_id); + }else if (is_list_node_type (ptr_type)) { +struct btf_field *field = meta->arg_list_head.field; + +mark_reg_graph_node (regs,BPF_REG_0 , &field->graph_root); + }else if (is_rbtree_node_type (ptr_type)) { +struct btf_field *field = meta->arg_rbtree_root.field; + +mark_reg_graph_node (regs,BPF_REG_0 , &field->graph_root); + }else if (meta->func_id == special_kfunc_list[KF_bpf_cast_to_kern_ctx]) { +mark_reg_known_zero (env, regs,BPF_REG_0 ); + regs[BPF_REG_0 ].type =PTR_TO_BTF_ID |PTR_TRUSTED ; + regs[BPF_REG_0 ].btf = desc_btf; + regs[BPF_REG_0 ].btf_id = meta->ret_btf_id; + }else if (meta->func_id == special_kfunc_list[KF_bpf_rdonly_cast]) { + ret_t =btf_type_by_id (desc_btf, meta->arg_constant.value); +if (!ret_t || !btf_type_is_struct (ret_t)) { +verbose (env, +"kfunc bpf_rdonly_cast type ID argument must be of a struct\n" ); +return -EINVAL ; + } + +mark_reg_known_zero (env, regs,BPF_REG_0 ); + regs[BPF_REG_0 ].type =PTR_TO_BTF_ID |PTR_UNTRUSTED ; + regs[BPF_REG_0 ].btf = desc_btf; + regs[BPF_REG_0 ].btf_id = meta->arg_constant.value; + }else if (meta->func_id == special_kfunc_list[KF_bpf_dynptr_slice] || + meta->func_id == special_kfunc_list[KF_bpf_dynptr_slice_rdwr]) { +enum bpf_type_flag type_flag =get_dynptr_type_flag (meta->initialized_dynptr.type); + +mark_reg_known_zero (env, regs,BPF_REG_0 ); + +if (!meta->arg_constant.found) { +verbose (env,"verifier internal error: bpf_dynptr_slice(_rdwr) no constant size\n" ); +return -EFAULT ; + } + + regs[BPF_REG_0 ].mem_size = meta->arg_constant.value; + +/* PTR_MAYBE_NULL will be added when is_kfunc_ret_null is checked */ + regs[BPF_REG_0 ].type =PTR_TO_MEM | type_flag; + +if (meta->func_id == special_kfunc_list[KF_bpf_dynptr_slice]) { + regs[BPF_REG_0 ].type |=MEM_RDONLY ; + }else { +/* this will set env->seen_direct_write to true */ +if (!may_access_direct_pkt_data (env,NULL ,BPF_WRITE )) { +verbose (env,"the prog does not allow writes to packet data\n" ); +return -EINVAL ; + } + } + +if (!meta->initialized_dynptr.id) { +verbose (env,"verifier internal error: no dynptr id\n" ); +return -EFAULT ; + } + regs[BPF_REG_0 ].dynptr_id = meta->initialized_dynptr.id; + +/* we don't need to set BPF_REG_0's ref obj id +* because packet slices are not refcounted (see +* dynptr_type_refcounted) +*/ + }else { + *done = false; + } + +return 0 ; +} +static int check_return_code (struct bpf_verifier_env *env,int regno,const char *reg_name); static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
@@ -13466,7 +13597,6 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,struct bpf_insn_aux_data *insn_aux;int err, insn_idx = *insn_idx_p;const struct btf_param *args; -const struct btf_type *ret_t;struct btf *desc_btf;/* skip for now, but return error when we find this in fixup_kfunc_call */
@@ -13685,165 +13815,16 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,__mark_reg_const_zero (env, ®s[BPF_REG_0 ]);mark_btf_func_reg_size (env,BPF_REG_0 , t->size); }else if (btf_type_is_ptr (t)) { - ptr_type =btf_type_skip_modifiers (desc_btf, t->type, &ptr_type_id); - -if (meta.btf == btf_vmlinux &&btf_id_set_contains (&special_kfunc_set, meta.func_id)) { -if (meta.func_id == special_kfunc_list[KF_bpf_obj_new_impl] || - meta.func_id == special_kfunc_list[KF_bpf_percpu_obj_new_impl]) { -struct btf_struct_meta *struct_meta; -struct btf *ret_btf; -u32 ret_btf_id; - -if (meta.func_id == special_kfunc_list[KF_bpf_obj_new_impl] && !bpf_global_ma_set) -return -ENOMEM ; - -if (((u64 )(u32 )meta.arg_constant.value) != meta.arg_constant.value) { -verbose (env,"local type ID argument must be in range [0, U32_MAX]\n" ); -return -EINVAL ; - } - - ret_btf = env->prog->aux->btf; - ret_btf_id = meta.arg_constant.value; +bool done = true; -/* This may be NULL due to user not supplying a BTF */ -if (!ret_btf) { -verbose (env,"bpf_obj_new/bpf_percpu_obj_new requires prog BTF\n" ); -return -EINVAL ; - } - - ret_t =btf_type_by_id (ret_btf, ret_btf_id); -if (!ret_t || !__btf_type_is_struct (ret_t)) { -verbose (env,"bpf_obj_new/bpf_percpu_obj_new type ID argument must be of a struct\n" ); -return -EINVAL ; - } - -if (meta.func_id == special_kfunc_list[KF_bpf_percpu_obj_new_impl]) { -if (ret_t->size >BPF_GLOBAL_PERCPU_MA_MAX_SIZE ) { -verbose (env,"bpf_percpu_obj_new type size (%d) is greater than %d\n" , - ret_t->size,BPF_GLOBAL_PERCPU_MA_MAX_SIZE ); -return -EINVAL ; - } - -if (!bpf_global_percpu_ma_set) { -mutex_lock (&bpf_percpu_ma_lock); -if (!bpf_global_percpu_ma_set) { -/* Charge memory allocated with bpf_global_percpu_ma to -* root memcg. The obj_cgroup for root memcg is NULL. -*/ - err =bpf_mem_alloc_percpu_init (&bpf_global_percpu_ma,NULL ); -if (!err) - bpf_global_percpu_ma_set = true; - } -mutex_unlock (&bpf_percpu_ma_lock); -if (err) -return err; - } - -mutex_lock (&bpf_percpu_ma_lock); - err =bpf_mem_alloc_percpu_unit_init (&bpf_global_percpu_ma, ret_t->size); -mutex_unlock (&bpf_percpu_ma_lock); -if (err) -return err; - } - - struct_meta =btf_find_struct_meta (ret_btf, ret_btf_id); -if (meta.func_id == special_kfunc_list[KF_bpf_percpu_obj_new_impl]) { -if (!__btf_type_is_scalar_struct (env, ret_btf, ret_t,0 )) { -verbose (env,"bpf_percpu_obj_new type ID argument must be of a struct of scalars\n" ); -return -EINVAL ; - } - -if (struct_meta) { -verbose (env,"bpf_percpu_obj_new type ID argument must not contain special fields\n" ); -return -EINVAL ; - } - } - -mark_reg_known_zero (env, regs,BPF_REG_0 ); - regs[BPF_REG_0 ].type =PTR_TO_BTF_ID |MEM_ALLOC ; - regs[BPF_REG_0 ].btf = ret_btf; - regs[BPF_REG_0 ].btf_id = ret_btf_id; -if (meta.func_id == special_kfunc_list[KF_bpf_percpu_obj_new_impl]) - regs[BPF_REG_0 ].type |=MEM_PERCPU ; - - insn_aux->obj_new_size = ret_t->size; - insn_aux->kptr_struct_meta = struct_meta; - }else if (meta.func_id == special_kfunc_list[KF_bpf_refcount_acquire_impl]) { -mark_reg_known_zero (env, regs,BPF_REG_0 ); - regs[BPF_REG_0 ].type =PTR_TO_BTF_ID |MEM_ALLOC ; - regs[BPF_REG_0 ].btf = meta.arg_btf; - regs[BPF_REG_0 ].btf_id = meta.arg_btf_id; - - insn_aux->kptr_struct_meta = -btf_find_struct_meta (meta.arg_btf, - meta.arg_btf_id); - }else if (is_list_node_type (ptr_type)) { -struct btf_field *field = meta.arg_list_head.field; - -mark_reg_graph_node (regs,BPF_REG_0 , &field->graph_root); - }else if (is_rbtree_node_type (ptr_type)) { -struct btf_field *field = meta.arg_rbtree_root.field; - -mark_reg_graph_node (regs,BPF_REG_0 , &field->graph_root); - }else if (meta.func_id == special_kfunc_list[KF_bpf_cast_to_kern_ctx]) { -mark_reg_known_zero (env, regs,BPF_REG_0 ); - regs[BPF_REG_0 ].type =PTR_TO_BTF_ID |PTR_TRUSTED ; - regs[BPF_REG_0 ].btf = desc_btf; - regs[BPF_REG_0 ].btf_id = meta.ret_btf_id; - }else if (meta.func_id == special_kfunc_list[KF_bpf_rdonly_cast]) { - ret_t =btf_type_by_id (desc_btf, meta.arg_constant.value); -if (!ret_t || !btf_type_is_struct (ret_t)) { -verbose (env, -"kfunc bpf_rdonly_cast type ID argument must be of a struct\n" ); -return -EINVAL ; - } - -mark_reg_known_zero (env, regs,BPF_REG_0 ); - regs[BPF_REG_0 ].type =PTR_TO_BTF_ID |PTR_UNTRUSTED ; - regs[BPF_REG_0 ].btf = desc_btf; - regs[BPF_REG_0 ].btf_id = meta.arg_constant.value; - }else if (meta.func_id == special_kfunc_list[KF_bpf_dynptr_slice] || - meta.func_id == special_kfunc_list[KF_bpf_dynptr_slice_rdwr]) { -enum bpf_type_flag type_flag =get_dynptr_type_flag (meta.initialized_dynptr.type); - -mark_reg_known_zero (env, regs,BPF_REG_0 ); - -if (!meta.arg_constant.found) { -verbose (env,"verifier internal error: bpf_dynptr_slice(_rdwr) no constant size\n" ); -return -EFAULT ; - } - - regs[BPF_REG_0 ].mem_size = meta.arg_constant.value; - -/* PTR_MAYBE_NULL will be added when is_kfunc_ret_null is checked */ - regs[BPF_REG_0 ].type =PTR_TO_MEM | type_flag; - -if (meta.func_id == special_kfunc_list[KF_bpf_dynptr_slice]) { - regs[BPF_REG_0 ].type |=MEM_RDONLY ; - }else { -/* this will set env->seen_direct_write to true */ -if (!may_access_direct_pkt_data (env,NULL ,BPF_WRITE )) { -verbose (env,"the prog does not allow writes to packet data\n" ); -return -EINVAL ; - } - } - -if (!meta.initialized_dynptr.id) { -verbose (env,"verifier internal error: no dynptr id\n" ); -return -EFAULT ; - } - regs[BPF_REG_0 ].dynptr_id = meta.initialized_dynptr.id; + ptr_type =btf_type_skip_modifiers (desc_btf, t->type, &ptr_type_id); + err =check_special_kfunc (env, &meta, regs, insn_aux, ptr_type, desc_btf, &done); +if (err) +return err; +if (done) + goto check_kfunc_ret; -/* we don't need to set BPF_REG_0's ref obj id -* because packet slices are not refcounted (see -* dynptr_type_refcounted) -*/ - }else { -verbose (env,"kernel function %s unhandled dynamic return type\n" , - meta.func_name); -return -EFAULT ; - } - }else if (btf_type_is_void (ptr_type)) { +if (btf_type_is_void (ptr_type)) {/* kfunc returning 'void *' is equivalent to returning scalar */ mark_reg_unknown (env, regs,BPF_REG_0 ); }else if (!__btf_type_is_struct (ptr_type)) {
@@ -13897,6 +13878,7 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, } } +check_kfunc_ret:if (is_kfunc_ret_null (&meta)) { regs[BPF_REG_0 ].type |=PTR_MAYBE_NULL ;/* For mark_ptr_or_null_reg, see 93c230e3f5bd6 */
@@ -13918,7 +13900,7 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,if (reg_may_point_to_spin_lock (®s[BPF_REG_0 ]) && !regs[BPF_REG_0 ].id) regs[BPF_REG_0 ].id = ++env->id_gen; }else if (btf_type_is_void (t)) { -if (meta.btf == btf_vmlinux &&btf_id_set_contains (&special_kfunc_set, meta.func_id)) { +if (meta.btf == btf_vmlinux) {if (meta.func_id == special_kfunc_list[KF_bpf_obj_drop_impl] || meta.func_id == special_kfunc_list[KF_bpf_percpu_obj_drop_impl]) { insn_aux->kptr_struct_meta =
Signature · 1 line
2.47.1
Re: [PATCH bpf-next v4 1/3] bpf: Remove special_kfunc_set from verifier
kernel/bpf/verifier.c static int fetch_kfunc_meta(struct bpf_verifier_env *env, · line 13428
Quoted 13 lines · Yonghong Song
kernel/bpf/verifier.c +12
-0+static int check_special_kfunc (struct bpf_verifier_env *env,struct bpf_kfunc_call_arg_meta *meta, +struct bpf_reg_state *regs,struct bpf_insn_aux_data *insn_aux, +const struct btf_type *ptr_type,struct btf *desc_btf,bool *done) +{ +const struct btf_type *ret_t; +int err =0 ; + +if (meta->btf != btf_vmlinux) { + *done = false; +return 0 ; + } +
I don't like 'bool *' approach, especially when it's not set in all branches, but implicitly initialized by the caller. Let's signal by returning 0, 1 or negative error instead. #define KFUNC_OK 1
kernel/bpf/verifier.c static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, · line 13825
Quoted 6 lines
kernel/bpf/verifier.c +6
-0+ ptr_type =btf_type_skip_modifiers (desc_btf, t->type, &ptr_type_id); + err =check_special_kfunc (env, &meta, regs, insn_aux, ptr_type, desc_btf, &done); +if (err) +return err; +if (done) + goto check_kfunc_ret;
This will be:
err = check_special_kfunc(..);
if (err) {
if (err < 0)
return err;
} else if (btf_type_is_void(ptr_type)) {Yonghong Song
reviewerPATCH v4 2/3 bpf: Warn with bpf_unreachable() kfunc maybe due to uninitialized variable
In reply to Yonghong SongMarc Suñé (Isovalent, part of Cisco) reported an issue where an uninitialized variable caused generating bpf prog binary code not working as expected. The reproducer is in [1] where the flags “-Wall -Werror” are enabled, but there is no warning as the compiler takes advantage of uninitialized variable to do aggressive optimization. The optimized code looks like below:
; {
0: bf 16 00 00 00 00 00 00 r6 = r1
; bpf_printk("Start");
1: 18 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 r1 = 0x0 ll
0000000000000008: R_BPF_64_64 .rodata
3: b4 02 00 00 06 00 00 00 w2 = 0x6
4: 85 00 00 00 06 00 00 00 call 0x6
; DEFINE_FUNC_CTX_POINTER(data)
5: 61 61 4c 00 00 00 00 00 w1 = *(u32 *)(r6 + 0x4c)
; bpf_printk("pre ipv6_hdrlen_offset");
6: 18 01 00 00 06 00 00 00 00 00 00 00 00 00 00 00 r1 = 0x6 ll
0000000000000030: R_BPF_64_64 .rodata
8: b4 02 00 00 17 00 00 00 w2 = 0x17
9: 85 00 00 00 06 00 00 00 call 0x6
<END>The verifier will report the following failure:
9: (85) call bpf_trace_printk#6
last insn is not an exit or jmp
The above verifier log does not give a clear hint about how to fix the problem and user may take quite some time to figure out that the issue is due to compiler taking advantage of uninitialized variable.
In llvm internals, uninitialized variable usage may generate 'unreachable' IR insn and these 'unreachable' IR insns may indicate uninitialized variable impact on code optimization. So far, llvm BPF backend ignores 'unreachable' IR hence the above code is generated. With clang21 patch [2], those 'unreachable' IR insn are converted to func bpf_unreachable(). In order to maintain proper control flow graph for bpf progs, [2] also adds an 'exit' insn after bpf_unreachable() if bpf_unreachable() is the last insn in the function.
The new code looks like:
; {
0: bf 16 00 00 00 00 00 00 r6 = r1
; bpf_printk("Start");
1: 18 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 r1 = 0x0 ll
0000000000000008: R_BPF_64_64 .rodata
3: b4 02 00 00 06 00 00 00 w2 = 0x6
4: 85 00 00 00 06 00 00 00 call 0x6
; DEFINE_FUNC_CTX_POINTER(data)
5: 61 61 4c 00 00 00 00 00 w1 = *(u32 *)(r6 + 0x4c)
; bpf_printk("pre ipv6_hdrlen_offset");
6: 18 01 00 00 06 00 00 00 00 00 00 00 00 00 00 00 r1 = 0x6 ll
0000000000000030: R_BPF_64_64 .rodata
8: b4 02 00 00 17 00 00 00 w2 = 0x17
9: 85 00 00 00 06 00 00 00 call 0x6
10: 85 10 00 00 ff ff ff ff call -0x1
0000000000000050: R_BPF_64_32 bpf_unreachable
11: 95 00 00 00 00 00 00 00 exit
<END>In kernel, a new kfunc bpf_unreachable() is added. During insn verification, any hit with bpf_unreachable() will result in verification failure. The kernel is able to provide better log message for debugging.
With llvm patch [2] and without this patch (no bpf_unreachable() kfunc for existing kernel), e.g., for old kernels, the verifier outputs
10: <invalid kfunc call>
kfunc 'bpf_unreachable' is referenced but wasn't resolved
Basically, kernel does not support bpf_unreachable() kfunc. This still didn't give clear signals about possible reason.
With llvm patch [2] and with this patch, the verifier outputs
10: (85) call bpf_unreachable#74479
unexpected bpf_unreachable() due to uninitialized variable?
It gives much better hints for verification failure.
[1] https://github.com/msune/clang_bpf/blob/main/Makefile#L3
[2] https://github.com/llvm/llvm-project/pull/131731
--- kernel/bpf/helpers.c | 5 +++++ kernel/bpf/verifier.c | 5 +++++ 2 files changed, 10 insertions(+)
kernel/bpf/helpers.c +5 -0
@@ -3273,6 +3273,10 @@ __bpf_kfunc void bpf_local_irq_restore(unsigned long *flags__irq_flag)local_irq_restore (*flags__irq_flag); } +__bpf_kfunc void bpf_unreachable(void) +{ +} +__bpf_kfunc_end_defs ();BTF_KFUNCS_START (generic_btf_ids)
@@ -3386,6 +3390,7 @@ BTF_ID_FLAGS(func, bpf_copy_from_user_dynptr, KF_SLEEPABLE) BTF_ID_FLAGS(func, bpf_copy_from_user_str_dynptr, KF_SLEEPABLE) BTF_ID_FLAGS(func, bpf_copy_from_user_task_dynptr, KF_SLEEPABLE | KF_TRUSTED_ARGS) BTF_ID_FLAGS(func, bpf_copy_from_user_task_str_dynptr, KF_SLEEPABLE | KF_TRUSTED_ARGS) +BTF_ID_FLAGS(func, bpf_unreachable)BTF_KFUNCS_END (common_btf_ids) static const struct btf_kfunc_id_set common_kfunc_set = {
kernel/bpf/verifier.c +5 -0
@@ -12105,6 +12105,7 @@ enum special_kfunc_type { KF_bpf_res_spin_unlock, KF_bpf_res_spin_lock_irqsave, KF_bpf_res_spin_unlock_irqrestore, + KF_bpf_unreachable, };BTF_ID_LIST (special_kfunc_list)
@@ -12170,6 +12171,7 @@ BTF_ID(func, bpf_res_spin_lock) BTF_ID(func, bpf_res_spin_unlock) BTF_ID(func, bpf_res_spin_lock_irqsave) BTF_ID(func, bpf_res_spin_unlock_irqrestore) +BTF_ID(func, bpf_unreachable)static bool is_kfunc_ret_null (struct bpf_kfunc_call_arg_meta *meta) {
@@ -13638,6 +13640,9 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,return err; }__mark_btf_func_reg_size (env, regs,BPF_REG_0 ,sizeof (u32)); + }else if (!insn->off && insn->imm == special_kfunc_list[KF_bpf_unreachable]) { +verbose (env,"unexpected bpf_unreachable() due to uninitialized variable?\n" ); +return -EFAULT ; }if (is_kfunc_destructive (&meta) && !capable (CAP_SYS_BOOT )) {
Signature · 1 line
2.47.1
kernel/bpf/helpers.c __bpf_kfunc void bpf_local_irq_restore(unsigned long *flags__irq_flag) · line 3279
Quoted 5 lines · Yonghong Song
kernel/bpf/helpers.c +4
-0+__bpf_kfunc void bpf_unreachable(void) +{ +} +
Eduard made a good point on LLVM diff that reserved functions have to start with double underscore.
Yonghong Song
reviewerPATCH v4 3/3 selftests/bpf: Add unit tests with bpf_unreachable() kfunc
In reply to Yonghong SongThe compiler support for bpf_unreachable() ([1]) will be in llvm21, but the current kernel will have a build failure with llvm21 ([2]). So all unit tests have explicit references to bpf_unreachable(). The test where bpf_unreachable() is generated by compiler will be provided later.
[1] https://github.com/llvm/llvm-project/pull/131731
[2] https://patchew.org/linux/[email protected]/
--- .../selftests/bpf/prog_tests/verifier.c | 2 + .../bpf/progs/verifier_bpf_unreachable.c | 61 +++++++++++++++++++ 2 files changed, 63 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/verifier_bpf_unreachable.c
tools/testing/selftests/bpf/prog_tests/verifier.c +2 -0
@@ -14,6 +14,7 @@#include "verifier_bounds_deduction_non_const.skel.h" #include "verifier_bounds_mix_sign_unsign.skel.h" #include "verifier_bpf_get_stack.skel.h" +#include "verifier_bpf_unreachable.skel.h" #include "verifier_bswap.skel.h" #include "verifier_btf_ctx_access.skel.h" #include "verifier_btf_unreliable_prog.skel.h"
@@ -148,6 +149,7 @@ void test_verifier_bounds_deduction(void) { RUN(verifier_bounds_deduction);void test_verifier_bounds_deduction_non_const (void ) {RUN (verifier_bounds_deduction_non_const); }void test_verifier_bounds_mix_sign_unsign (void ) {RUN (verifier_bounds_mix_sign_unsign); }void test_verifier_bpf_get_stack (void ) {RUN (verifier_bpf_get_stack); } +void test_verifier_bpf_unreachable (void ) {RUN (verifier_bpf_unreachable); }void test_verifier_bswap (void ) {RUN (verifier_bswap); }void test_verifier_btf_ctx_access (void ) {RUN (verifier_btf_ctx_access); }void test_verifier_btf_unreliable_prog (void ) {RUN (verifier_btf_unreliable_prog); }
tools/testing/selftests/bpf/progs/verifier_bpf_unreachable.c new file +61 -0
61 lines not on this page · raw message
Signature · 1 line
2.47.1