bpf: Implement two type cast kfuncs
Subsystems (4)
v3 cover bpf: Implement two type cast kfuncs
Currenty, a non-tracing bpf program typically has a single 'context' argument with predefined uapi struct type. Following these uapi struct, user is able to access other fields defined in uapi header. Inside the kernel, the user-seen 'context' argument is replaced with 'kernel context' (or 'kctx' in short) which can access more information than what uapi header provides. To access other info not in uapi header, people typically do two things:
(1). extend uapi to access more fields rooted from 'context'. (2). use bpf_probe_read_kernl() helper to read particular field based on
kctx.
Using (1) needs uapi change and using (2) makes code more complex since direct memory access is not allowed.
There are already a few instances trying to access more information from kctx:
. trying to access some fields from perf_event kctx ([1]). . trying to access some fields from xdp kctx ([2]).
This patch set tried to allow direct memory access for kctx fields by introducing bpf_cast_to_kern_ctx() kfunc.
Martin mentioned a use case like type casting below:
#define skb_shinfo(SKB) ((struct skb_shared_info *)(skb_end_pointer(SKB)))
basically a 'unsigned char *" casted to 'struct skb_shared_info *'. This patch set tries to support such a use case as well with bpf_rdonly_cast().
For the patch series, Patch 1 added support for a kfunc available to all prog types. Patch 2 added bpf_cast_to_kern_ctx() kfunc. Patch 3 added bpf_rdonly_cast() kfunc. Patch 4 added a few positive and negative tests.
[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://lore.kernel.org/bpf/[email protected]/
Changelog:
v2 -> v3:
- rebase on top of bpf-next (for merging conflicts)
- add the selftest to s390x deny list
rfcv1 -> v2:
- break original one kfunc into two.
- add missing error checks and error logs.
- adapt to the new conventions in
https://lore.kernel.org/all/[email protected]/
for example, with __ign and __k suffix.
- added support in fixup_kfunc_call() to replace kfunc calls with a single mov.
Yonghong Song (4):
bpf: Add support for kfunc set with common btf_ids
bpf: Add a kfunc to type cast from bpf uapi ctx to kernel ctx
bpf: Add a kfunc for generic type cast
bpf: Add type cast unit tests
include/linux/btf.h | 5 + kernel/bpf/btf.c | 33 +++++ kernel/bpf/helpers.c | 24 +++- kernel/bpf/verifier.c | 45 ++++++- tools/testing/selftests/bpf/DENYLIST.s390x | 1 + .../selftests/bpf/prog_tests/type_cast.c | 114 ++++++++++++++++++ tools/testing/selftests/bpf/progs/type_cast.c | 83 +++++++++++++ 7 files changed, 303 insertions(+), 2 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/type_cast.c create mode 100644 tools/testing/selftests/bpf/progs/type_cast.c
Signature · 1 line
2.30.2
PATCH v3 1/4 bpf: Add support for kfunc set with common btf_ids
Later on, we will introduce kfuncs bpf_cast_to_kern_ctx() and bpf_rdonly_cast() which apply to all program types. Currently kfunc set only supports individual prog types. This patch added support for kfunc applying to all program types.
--- kernel/bpf/btf.c | 8 ++++++++ kernel/bpf/helpers.c | 12 +++++++++++- 2 files changed, 19 insertions(+), 1 deletion(-)
kernel/bpf/btf.c +8 -0
@@ -199,6 +199,7 @@ DEFINE_IDR(btf_idr);DEFINE_SPINLOCK (btf_idr_lock);enum btf_kfunc_hook { +BTF_KFUNC_HOOK_COMMON ,BTF_KFUNC_HOOK_XDP ,BTF_KFUNC_HOOK_TC ,BTF_KFUNC_HOOK_STRUCT_OPS ,
@@ -7531,6 +7532,8 @@ static u32 *__btf_kfunc_id_set_contains(const struct btf *btf,static int bpf_prog_type_to_kfunc_hook (enum bpf_prog_type prog_type) {switch (prog_type) { +case BPF_PROG_TYPE_UNSPEC : +return BTF_KFUNC_HOOK_COMMON ;case BPF_PROG_TYPE_XDP :return BTF_KFUNC_HOOK_XDP ;case BPF_PROG_TYPE_SCHED_CLS :
@@ -7559,6 +7562,11 @@ u32 *btf_kfunc_id_set_contains(const struct btf *btf, u32 kfunc_btf_id) {enum btf_kfunc_hook hook; +u32 *kfunc_flags; + + kfunc_flags =__btf_kfunc_id_set_contains (btf,BTF_KFUNC_HOOK_COMMON , kfunc_btf_id); +if (kfunc_flags) +return kfunc_flags; hook =bpf_prog_type_to_kfunc_hook (prog_type);return __btf_kfunc_id_set_contains (btf, hook, kfunc_btf_id);
kernel/bpf/helpers.c +11 -1
@@ -1903,10 +1903,19 @@ static const struct btf_kfunc_id_set generic_kfunc_set = { .set = &generic_btf_ids, }; +BTF_ID_LIST (generic_dtor_ids) BTF_ID(struct, task_struct)BTF_ID (func, bpf_task_release) +BTF_SET8_START (common_btf_ids) +BTF_SET8_END (common_btf_ids) + +static const struct btf_kfunc_id_set common_kfunc_set = { + .owner =THIS_MODULE , + .set = &common_btf_ids, +}; +static int __init kfunc_init (void ) {int ret;
@@ -1920,9 +1929,10 @@ static int __init kfunc_init(void) ret =register_btf_kfunc_id_set (BPF_PROG_TYPE_TRACING , &generic_kfunc_set); ret = ret ?:register_btf_kfunc_id_set (BPF_PROG_TYPE_SCHED_CLS , &generic_kfunc_set); ret = ret ?:register_btf_kfunc_id_set (BPF_PROG_TYPE_STRUCT_OPS , &generic_kfunc_set); -return ret ?:register_btf_id_dtor_kfuncs (generic_dtors, + ret = ret ?:register_btf_id_dtor_kfuncs (generic_dtors,ARRAY_SIZE (generic_dtors),THIS_MODULE ); +return ret ?:register_btf_kfunc_id_set (BPF_PROG_TYPE_UNSPEC , &common_kfunc_set); }late_initcall (kfunc_init);
Signature · 1 line
2.30.2
PATCH v3 2/4 bpf: Add a kfunc to type cast from bpf uapi ctx to kernel ctx
In reply to Yonghong SongImplement bpf_cast_to_kern_ctx() kfunc which does a type cast of a uapi ctx object to the corresponding kernel ctx. Previously if users want to access some data available in kctx but not in uapi ctx, bpf_probe_read_kernel() helper is needed. The introduction of bpf_cast_to_kern_ctx() allows direct memory access which makes code simpler and easier to understand.
--- include/linux/btf.h | 5 +++++ kernel/bpf/btf.c | 25 +++++++++++++++++++++++++ kernel/bpf/helpers.c | 6 ++++++ kernel/bpf/verifier.c | 21 +++++++++++++++++++++ 4 files changed, 57 insertions(+)
include/linux/btf.h +5 -0
@@ -487,6 +487,7 @@ const struct btf_member * btf_get_prog_ctx_type(struct bpf_verifier_log *log, const struct btf *btf, const struct btf_type *t, enum bpf_prog_type prog_type, int arg); +int get_kern_ctx_btf_id (struct bpf_verifier_log *log,enum bpf_prog_type prog_type);bool btf_types_are_same (const struct btf *btf1,u32 id1,const struct btf *btf2,u32 id2);#else
@@ -531,6 +532,10 @@ btf_get_prog_ctx_type(struct bpf_verifier_log *log, const struct btf *btf, {return NULL ; } +static inline int get_kern_ctx_btf_id (struct bpf_verifier_log *log, +enum bpf_prog_type prog_type) { +return -EINVAL ; +}static inline bool btf_types_are_same (const struct btf *btf1,u32 id1,const struct btf *btf2,u32 id2) {
kernel/bpf/btf.c +25 -0
@@ -5603,6 +5603,31 @@ static int btf_translate_to_vmlinux(struct bpf_verifier_log *log,return kern_ctx_type->type; } +int get_kern_ctx_btf_id (struct bpf_verifier_log *log,enum bpf_prog_type prog_type) +{ +const struct btf_member *kctx_member; +const struct btf_type *conv_struct; +const struct btf_type *kctx_type; +u32 kctx_type_id; + + conv_struct = bpf_ctx_convert.t; +if (!conv_struct) { +bpf_log (log,"btf_vmlinux is malformed\n" ); +return -EINVAL ; + } + +/* get member for kernel ctx type */ + kctx_member =btf_type_member (conv_struct) + bpf_ctx_convert_map[prog_type] *2 +1 ; + kctx_type_id = kctx_member->type; + kctx_type =btf_type_by_id (btf_vmlinux, kctx_type_id); +if (!btf_type_is_struct (kctx_type)) { +bpf_log (log,"kern ctx type id %u is not a struct\n" , kctx_type_id); +return -EINVAL ; + } + +return kctx_type_id; +} +BTF_ID_LIST (bpf_ctx_convert_btf_id) BTF_ID(struct, bpf_ctx_convert)
kernel/bpf/helpers.c +6 -0
@@ -1881,6 +1881,11 @@ void bpf_task_release(struct task_struct *p)put_task_struct_rcu_user (p); } +void *bpf_cast_to_kern_ctx (void *obj) +{ +return obj; +} +__diag_pop ();BTF_SET8_START (generic_btf_ids)
@@ -1909,6 +1914,7 @@ BTF_ID(struct, task_struct) BTF_ID(func, bpf_task_release)BTF_SET8_START (common_btf_ids) +BTF_ID_FLAGS (func ,bpf_cast_to_kern_ctx )BTF_SET8_END (common_btf_ids) static const struct btf_kfunc_id_set common_kfunc_set = {
kernel/bpf/verifier.c +21 -0
@@ -8151,6 +8151,7 @@ enum special_kfunc_type { KF_bpf_list_push_back, KF_bpf_list_pop_front, KF_bpf_list_pop_back, + KF_bpf_cast_to_kern_ctx, };BTF_SET_START (special_kfunc_set)
@@ -8160,6 +8161,7 @@ BTF_ID(func, bpf_list_push_front) BTF_ID(func, bpf_list_push_back) BTF_ID(func, bpf_list_pop_front) BTF_ID(func, bpf_list_pop_back) +BTF_ID(func, bpf_cast_to_kern_ctx)BTF_SET_END (special_kfunc_set)BTF_ID_LIST (special_kfunc_list)
@@ -8169,6 +8171,7 @@ BTF_ID(func, bpf_list_push_front) BTF_ID(func, bpf_list_push_back) BTF_ID(func, bpf_list_pop_front) BTF_ID(func, bpf_list_pop_back) +BTF_ID(func, bpf_cast_to_kern_ctx) static enum kfunc_ptr_arg_type get_kfunc_ptr_arg_type(struct bpf_verifier_env *env,
@@ -8182,6 +8185,9 @@ get_kfunc_ptr_arg_type(struct bpf_verifier_env *env,struct bpf_reg_state *reg = ®s[regno];bool arg_mem_size = false; +if (meta->func_id == special_kfunc_list[KF_bpf_cast_to_kern_ctx]) +return KF_ARG_PTR_TO_CTX ; +/* In this function, we verify the kfunc's BTF as per the argument type, * leaving the rest of the verification with respect to the register * type to our caller. When a set of conditions hold in the BTF type of
@@ -8668,6 +8674,13 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_verbose (env,"arg#%d expected pointer to ctx, but got %s\n" , i,btf_type_str (t));return -EINVAL ; } + +if (meta->func_id == special_kfunc_list[KF_bpf_cast_to_kern_ctx]) { + ret =get_kern_ctx_btf_id (&env->log,resolve_prog_type (env->prog)); +if (ret <0 ) +return -EINVAL ; + meta->arg_constant.value = ret; + }break ;case KF_ARG_PTR_TO_ALLOC_BTF_ID :if (reg->type != (PTR_TO_BTF_ID |MEM_ALLOC )) {
@@ -8919,6 +8932,11 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, regs[BPF_REG_0 ].btf = field->list_head.btf; regs[BPF_REG_0 ].btf_id = field->list_head.value_btf_id; regs[BPF_REG_0 ].off = field->list_head.node_offset; + }else if (meta.func_id == special_kfunc_list[KF_bpf_cast_to_kern_ctx]) { +mark_reg_known_zero (env, regs,BPF_REG_0 ); + regs[BPF_REG_0 ].type =PTR_TO_BTF_ID ; + regs[BPF_REG_0 ].btf = desc_btf; + regs[BPF_REG_0 ].btf_id = meta.arg_constant.value; }else {verbose (env,"kernel function %s unhandled dynamic return type\n" , meta.func_name);
@@ -15172,6 +15190,9 @@ static int fixup_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, insn_buf[1 ] = addr[1 ]; insn_buf[2 ] = *insn; *cnt =3 ; + }else if (desc->func_id == special_kfunc_list[KF_bpf_cast_to_kern_ctx]) { + insn_buf[0 ] =BPF_MOV64_REG (BPF_REG_0 ,BPF_REG_1 ); + *cnt =1 ; }return 0 ; }
Signature · 1 line
2.30.2
Re: [PATCH bpf-next v3 2/4] bpf: Add a kfunc to type cast from bpf uapi ctx to kernel ctx
Quoted 48 lines · Yonghong Song
Implement
bpf_cast_to_kern_ctx()kfunc which does a type cast of a uapi ctx object to the corresponding kernel ctx. Previously if users want to access some data available in kctx but not in uapi ctx,bpf_probe_read_kernel()helper is needed. The introduction ofbpf_cast_to_kern_ctx()allows direct memory access which makes code simpler and easier to understand.Signed-off-by Yonghong Song <[email protected]>--- include/linux/btf.h | 5 +++++ kernel/bpf/btf.c | 25 +++++++++++++++++++++++++ kernel/bpf/helpers.c | 6 ++++++ kernel/bpf/verifier.c | 21 +++++++++++++++++++++ 4 files changed, 57 insertions(+)include/linux/btf.h +5
-0@@ -487,6 +487,7 @@ const struct btf_member * btf_get_prog_ctx_type(struct bpf_verifier_log *log, const struct btf *btf, const struct btf_type *t, enum bpf_prog_type prog_type, int arg); +int get_kern_ctx_btf_id (struct bpf_verifier_log *log,enum bpf_prog_type prog_type);bool btf_types_are_same (const struct btf *btf1,u32 id1,const struct btf *btf2,u32 id2);#else @@ -531,6 +532,10 @@ btf_get_prog_ctx_type(struct bpf_verifier_log *log, const struct btf *btf, {return NULL ; } +static inline int get_kern_ctx_btf_id (struct bpf_verifier_log *log, +enum bpf_prog_type prog_type) { +return -EINVAL ; +}static inline bool btf_types_are_same (const struct btf *btf1,u32 id1,const struct btf *btf2,u32 id2) {kernel/bpf/btf.c +2
-0@@ -5603,6 +5603,31 @@ static int btf_translate_to_vmlinux(struct bpf_verifier_log *log,return kern_ctx_type->type; } +int get_kern_ctx_btf_id (struct bpf_verifier_log *log,enum bpf_prog_type prog_type) +{
kernel/bpf/btf.c static int btf_translate_to_vmlinux(struct bpf_verifier_log *log, · line 5617
Quoted 10 lines
kernel/bpf/btf.c +10
-0+const struct btf_member *kctx_member; +const struct btf_type *conv_struct; +const struct btf_type *kctx_type; +u32 kctx_type_id; + + conv_struct = bpf_ctx_convert.t; +if (!conv_struct) { +bpf_log (log,"btf_vmlinux is malformed\n" ); +return -EINVAL ; + }
same comments as in v2.
PATCH v3 3/4 bpf: Add a kfunc for generic type cast
In reply to Yonghong SongImplement bpf_rdonly_cast() which tries to cast the object
to a specified type. This tries to support use case like below:
#define skb_shinfo(SKB) ((struct skb_shared_info *)(skb_end_pointer(SKB)))
where skb_end_pointer(SKB) is a 'unsigned char *' and needs to be casted to 'struct skb_shared_info *'.
The signature of bpf_rdonly_cast() looks like
void *bpf_rdonly_cast(void *obj, __u32 btf_id)
The function returns the same 'obj' but with PTR_TO_BTF_ID with btf_id. The verifier will ensure btf_id being a struct type.
Since the supported type cast may not reflect what the 'obj' represents, the returned btf_id is marked as PTR_UNTRUSTED, so the return value and subsequent pointer chasing cannot be used as helper/kfunc arguments.
--- kernel/bpf/helpers.c | 6 ++++++ kernel/bpf/verifier.c | 26 ++++++++++++++++++++++++-- 2 files changed, 30 insertions(+), 2 deletions(-)
kernel/bpf/helpers.c +6 -0
@@ -1886,6 +1886,11 @@ void *bpf_cast_to_kern_ctx(void *obj)return obj; } +void *bpf_rdonly_cast (void *obj__ign,u32 btf_id__k) +{ +return obj__ign; +} +__diag_pop ();BTF_SET8_START (generic_btf_ids)
@@ -1915,6 +1920,7 @@ BTF_ID(func, bpf_task_release)BTF_SET8_START (common_btf_ids)BTF_ID_FLAGS (func ,bpf_cast_to_kern_ctx ) +BTF_ID_FLAGS (func, bpf_rdonly_cast)BTF_SET8_END (common_btf_ids) static const struct btf_kfunc_id_set common_kfunc_set = {
kernel/bpf/verifier.c +24 -2
@@ -8152,6 +8152,7 @@ enum special_kfunc_type { KF_bpf_list_pop_front, KF_bpf_list_pop_back, KF_bpf_cast_to_kern_ctx, + KF_bpf_rdonly_cast, };BTF_SET_START (special_kfunc_set)
@@ -8162,6 +8163,7 @@ BTF_ID(func, bpf_list_push_back) BTF_ID(func, bpf_list_pop_front) BTF_ID(func, bpf_list_pop_back) BTF_ID(func, bpf_cast_to_kern_ctx) +BTF_ID(func, bpf_rdonly_cast)BTF_SET_END (special_kfunc_set)BTF_ID_LIST (special_kfunc_list)
@@ -8172,6 +8174,7 @@ BTF_ID(func, bpf_list_push_back) BTF_ID(func, bpf_list_pop_front) BTF_ID(func, bpf_list_pop_back) BTF_ID(func, bpf_cast_to_kern_ctx) +BTF_ID(func, bpf_rdonly_cast) static enum kfunc_ptr_arg_type get_kfunc_ptr_arg_type(struct bpf_verifier_env *env,
@@ -8808,6 +8811,7 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,u32 i, nargs, func_id, ptr_type_id;int err, insn_idx = *insn_idx_p;const struct btf_param *args; +const struct btf_type *ret_t;struct btf *desc_btf;u32 *kfunc_flags;
@@ -8887,7 +8891,6 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,if (meta.btf == btf_vmlinux &&btf_id_set_contains (&special_kfunc_set, meta.func_id)) {if (meta.func_id == special_kfunc_list[KF_bpf_obj_new_impl]) { -const struct btf_type *ret_t;struct btf *ret_btf;u32 ret_btf_id;
@@ -8937,6 +8940,24 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, regs[BPF_REG_0 ].type =PTR_TO_BTF_ID ; regs[BPF_REG_0 ].btf = desc_btf; regs[BPF_REG_0 ].btf_id = meta.arg_constant.value; + }else if (meta.func_id == special_kfunc_list[KF_bpf_rdonly_cast]) { +if (!capable (CAP_PERFMON )) { +verbose (env, +"kfunc bpf_rdonly_cast requires CAP_PERFMON capability\n" ); +return -EACCES ; + } + + ret_t =btf_type_by_id (desc_btf, meta.arg_constant.value); +if (!ret_t || !btf_type_is_struct (ret_t)) { +verbose (env, +"kfunc bpf_rdonly_cast type ID argument must be of a struct\n" ); +return -EINVAL ; + } + +mark_reg_known_zero (env, regs,BPF_REG_0 ); + regs[BPF_REG_0 ].type =PTR_TO_BTF_ID |PTR_UNTRUSTED ; + regs[BPF_REG_0 ].btf = desc_btf; + regs[BPF_REG_0 ].btf_id = meta.arg_constant.value; }else {verbose (env,"kernel function %s unhandled dynamic return type\n" , meta.func_name);
@@ -15190,7 +15211,8 @@ static int fixup_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, insn_buf[1 ] = addr[1 ]; insn_buf[2 ] = *insn; *cnt =3 ; - }else if (desc->func_id == special_kfunc_list[KF_bpf_cast_to_kern_ctx]) { + }else if (desc->func_id == special_kfunc_list[KF_bpf_cast_to_kern_ctx] || + desc->func_id == special_kfunc_list[KF_bpf_rdonly_cast]) { insn_buf[0 ] =BPF_MOV64_REG (BPF_REG_0 ,BPF_REG_1 ); *cnt =1 ; }
Signature · 1 line
2.30.2
PATCH v3 4/4 bpf: Add type cast unit tests
In reply to Yonghong SongThree tests are added. One is from John Fastabend ({1]) which tests tracing style access for xdp program from the kernel ctx. Another is a tc test to test both kernel ctx tracing style access and explicit non-ctx type cast. The third one is for negative tests including two tests, a tp_bpf test where the bpf_rdonly_cast() returns a untrusted ptr which cannot be used as helper argument, and a tracepoint test where the kernel ctx is a u64.
Also added the test to DENYLIST.s390x since s390 does not currently support calling kernel functions in JIT mode.
[1] https://lore.kernel.org/bpf/[email protected]/
--- tools/testing/selftests/bpf/DENYLIST.s390x | 1 + .../selftests/bpf/prog_tests/type_cast.c | 114 ++++++++++++++++++ tools/testing/selftests/bpf/progs/type_cast.c | 83 +++++++++++++ 3 files changed, 198 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/type_cast.c create mode 100644 tools/testing/selftests/bpf/progs/type_cast.c
tools/testing/selftests/bpf/DENYLIST.s390x +1 -0
@@ -71,6 +71,7 @@ trace_printk # trace_printk__load unexpected error: trace_vprintk # trace_vprintk__open_and_load unexpected error: -9 (?) tracing_struct # failed to auto-attach: -524 (trampoline) trampoline_count # prog 'prog1': failed to attach: ERROR: strerror_r(-524)=22 (trampoline) +type_cast # JIT does not support calling kernel function unpriv_bpf_disabled # fentry user_ringbuf # failed to find kernel BTF type ID of '__s390x_sys_prctl': -3 (?) verif_stats # trace_vprintk__open_and_load unexpected error: -9 (?)
tools/testing/selftests/bpf/prog_tests/type_cast.c new file +114 -0
114 lines not on this page · raw message
tools/testing/selftests/bpf/progs/type_cast.c new file +83 -0
83 lines not on this page · raw message
Signature · 1 line
2.30.2